Parsing with context embeddings

Ömer Kırnap, Berkay Furkan Önder and Deniz Yuret. In Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, Vancouver, 2017. (PDF, poster, presentation, results, 2015 version, 2018 version, Ömer's MS thesis).

Abstract. We introduce context embeddings, dense vectors derived from a language model that represent the left/right context of a word instance, and demonstrate that context embeddings significantly improve the accuracy of our transition based parser. Our model consists of a bidirectional LSTM (BiLSTM) based language model that is pre-trained to predict words in plain text, and a multi-layer perceptron (MLP) decision model that uses features from the language model to predict the correct actions for an ArcHybrid transition based parser. We participated in the CoNLL 2017 UD Shared Task as the ``Koç University'' team and our system was ranked 7th out of 33 systems that parsed 81 treebanks in 49 languages.

Full post...