Knet-the-Julia-dope: An interactive book on deep learning.

Written by Manuel Antonio Morales (@moralesq). This repo is the Julia translation of the mxnet-the-straight-dope repo, a collection of notebooks designed to teach deep learning, MXNet, and the gluon interface. This project grew out of the MIT course 6.338 Modern Numerical Computing with Julia taught by professor Alan Edelman. Our main objectives are:

• Introduce the Julia language and its main packages in the context of deep learning
  
• Introduce Julia's package Knet: an alternative/complementary option to MXNet
  
• Leverage the strengths of Jupyter notebooks to present prose, graphics, equations, and code together in one place

Wasserstein GAN: a Julia/Knet implementation

Written by Cem Eteke (@ceteke). This repository contains implementation of WGAN and DCGAN in Julia using Knet. Here is a detailed report about WGAN.

Relational networks: a Julia/Knet implementation

Written by Erenay Dayanık (@ereday). Knet implementation of "A simple neural network module for relational reasoning" by Santoro et al. (2017). (Relational Networks, arXiv:1706.01427, blog post)

Fast multidimensional reduction and broadcast operations on GPU for machine learning

Doğa Dikbayır, Enis Berk Çoban, İlker Kesen, Deniz Yuret, Didem Unat. Concurrency and Computation: Practice and Experience. 2018. (PDF). Abstract: Reduction and broadcast operations are commonly used in machine learning algorithms for different purposes. They widely appear in the calculation of the gradient values of a loss function, which are one of the core structures of neural networks. Both operations are implemented naively in many libraries usually for scalar reduction or broadcast; however, to our knowledge, there are no optimized multidimensional implementations available. This fact limits the performance of machine learning models requiring these operations to be performed on tensors. In this work, we address the problem and propose two new strategies that extend the existing implementations to perform on tensors. We introduce formal definitions of both operations using tensor notations, investigate their mathematical properties, and exploit these properties to provide an efficient solution for each. We implement our parallel strategies and test them on a CUDA enabled Tesla K40 m GPU accelerator. Our performant implementations achieve up to 75% of the peak device memory bandwidth on different tensor sizes and dimensions. Significant speedups against the implementations available in the Knet Deep Learning framework are also achieved for both operations.

Neural Style Transfer: a Julia notebook

Written by Cemil Cengiz (@cemilcengiz).

This notebook implements deep CNN based image style transfer algorithm from "Image Style Transfer Using Convolutional Neural Networks" (Gatys et al., CVPR 2016). The proposed technique takes two images as input, i.e. a content image (generally a photograph) and a style image (generally an artwork painting). Then, it produces an output image such that the content(objects in the image) resembles the "content image" whereas the style i.e. the texture is similar to the "style image". In order words, it re-draws the "content image" using the artistic style of the "style image".

The images below show an original photograph followed by two different styles applied by the network.

Happy birthday Raymond Smullyan

A mathematician friend of mine recently told me of a mathematician friend of his who everyday "takes a nap". Now, I never take naps. But I often fall asleep while reading -- which is very different from deliberately taking a nap! I am far more like my dogs Peekaboo, Peekatoo and Trixie than like my mathematician friend once removed. These dogs never take naps; they merely fall asleep. They fall asleep wherever and whenever they choose (which, incidentally is most of the time!). Thus these dogs are true sages.

I think this is all that Chinese philosophy is really about; the rest is mere elaboration!

Raymond Smullyan, The Tao is Silent (1977)

A new dataset and model for learning to understand navigational instructions

Ozan Arkan Can, Deniz Yuret (2018). arXiv:1805.07952. (PDF).

Abstract: In this paper, we present a state-of-the-art model and introduce a new dataset for grounded language learning. Our goal is to develop a model that can learn to follow new instructions given prior instruction-perception-action examples. We based our work on the SAIL dataset which consists of navigational instructions and actions in a maze-like environment. The new model we propose achieves the best results to date on the SAIL dataset by using an improved perceptual component that can represent relative positions of objects. We also analyze the problems with the SAIL dataset regarding its size and balance. We argue that performance on a small, fixed-size dataset is no longer a good measure to differentiate state-of-the-art models. We introduce SAILx, a synthetic dataset generator, and perform experiments where the size and balance of the dataset are controlled.

Tutorial: Deep Learning with Julia/Knet

Tutorial at Qatar Computing Research Institute, May 13, 2018. Thanks to Dr. Sanjay Chawla for the invitation.

• Introductory slides from JuliaCon 2017.
• MNIST notebook: Loading and visualizing the data.
• Linear notebook: Training, evaluating and visualizing a linear model.
• MLP notebook: Multi-layer perceptron, overfitting, regularization, dropout.
• CNN notebook: Convolutional networks.

Deep Learning in NLP: A Brief History

Panel presentation at the International Symposium on Brain and Cognitive Science (ISBCS 2018)