May 23, 2017

JuliaCon 2017, Berkeley, June 20-24

I am giving a talk at JuliaCon introducing Knet on Wednesday, June 21, 2017, 4:16pm - 4:52pm, East Pauley Pauley Ballroom, Berkeley, CA.
Full post...

May 17, 2017

Congratulations to the Koç parsing team

Our neural net based dependency parser was number 7 overall out of 33 teams participating in the CoNLL 2017 Shared Task "Multilingual Parsing from Raw Text to Universal Dependencies" in which participating teams had to parse 68 corpora in 50 languages. I would like to thank Ömer Kırnap and Berkay Furkan Önder for their contributions and all-nighters.
Full post...

April 28, 2017

The third deep learning revolution

The first revolution took place 1958-1969.
  • We figured out how to train perceptrons.
  • We proved the perceptron convergence theorem.
  • Interest waned after a book (Perceptrons) written by mathematicians.
The second revolution took place 1986-1995.
  • We figured out how to train multi-layer perceptrons.
  • We proved the universal approximation theorem.
  • Interest waned after a book (SLT) written by mathematicians.
The third revolution started in 2012.
  • We figured out how to train deep nets.
  • A mathematician will write a book around 2022.
  • The fourth revolution will not start until 2036 :)

Full post...

February 22, 2017

Overfitting, underfitting, regularization, dropout

Here is an IJulia notebook demonstrating overfitting, underfitting, regularization and dropout in Knet for my machine learning class.
Full post...