December 29, 2020

Ulaş Sert, M.S. 2020

Contact info: LinkedIn
M.S Thesis: Training a Bridge Bidding Agent using Minimal Feature Engineering and Deep Reinforcement Learning, Koç University, Department of Computer Engineering. December 2020. (PDF, Presentation, Code).

Thesis Abstract:
The game of contract bridge, or just bridge, is a four-player imperfect information card game where two partnerships of two players compete against each other. It has two main phases: bidding and play. While the computer players have approached human-level performance two decades ago in the playing phase, bidding is still a very challenging problem. This makes bridge one of the last popular games where computers still lag behind the expert human-level performance. During bidding, players only know their own cards while participating in a public auction. Performing well in this phase requires the players to figure out how to communicate with their partners using the limited vocabulary of bids to decide on a joint contract. This communication is restricted by the strict ordering of legal bids and can be negatively interfered by bids made by the opponent partnership. In this thesis, we experiment with several novel architectures with minimal feature engineering and evaluate them by using supervised training over a data set of expert-level human games. After that, we further study different forms of deep reinforcement learning to refine the resulting model by simulated gameplay. Lastly, we propose an oracle evaluation metric that can measure the quality of any bidding sequence with respect to the game-theoretical optimum.

Full post...

December 12, 2020

CRAFT: A Benchmark for Causal Reasoning About Forces and inTeractions

Tayfun Ates, Muhammed Samil Atesoglu, Cagatay Yigit, Ilker Kesen, Mert Kobas, Erkut Erdem, Aykut Erdem, Tilbe Goksun, Deniz Yuret. Shared Visual Representations in Human and Machine Intelligence (SVRHM 2020). NeurIPS Workshop. (PDF, Presentation)

Abstract: Recent advances in Artificial Intelligence and deep learning have revived the interest in studying the gap between the reasoning capabilities of humans and machines. In this ongoing work, we introduce CRAFT, a new visual question answering dataset that requires causal reasoning about physical forces and object interactions. It contains 38K video and question pairs that are generated from 3K videos from 10 different virtual environments, containing different number of objects in motion that interact with each other. Two question categories from CRAFT include previously studied descriptive and counterfactual questions. Besides, inspired by the theory of force dynamics from the field of human cognitive psychology, we introduce new question categories that involve understanding the intentions of objects through the notions of cause, enable, and prevent. Our preliminary results demonstrate that even though these tasks are very intuitive for humans, the implemented baselines could not cope with the underlying challenges.

Full post...

June 09, 2020

Cemil Cengiz, M.S. 2020

Current position: Senior Machine Learning Engineer at OBSS. (Linkedin, Homepage).
M.S Thesis: Improving Generalization in Natural Language Inference by Joint Training with Semantic Role Labeling, Koç University, Department of Computer Engineering. June 2020. (PDF, Presentation).
Publications: BibTeX

Thesis Abstract:
Recently, end-to-end models have achieved near-human performance on natural language inference (NLI) datasets. However, they show low generalization on out-of-distribution evaluation sets since they tend to learn shallow heuristics due to the biases in the training datasets. The performance decreases dramatically on diagnostic sets measuring compositionality or robustness against simple heuristics. Existing solutions for this problem employ dataset augmentation by extending the training dataset with examples from the evaluated adversarial categories. However, that approach has the drawbacks of being applicable to only a limited set of adversaries and at worst hurting the model performance on other adversaries not included in the augmentation set. Instead, our proposed solution is to improve sentence understanding (hence out-of-distribution generalization) with joint learning of explicit semantics. In this thesis, we show that a BERT based model trained jointly on English semantic role labeling (SRL) and NLI achieves significantly higher performance on external evaluation sets measuring generalization performance.

Full post...

May 13, 2020

Berkay Furkan Önder, M.S. 2020

Contact info: Email, GitHub.
M.S Thesis: Effect Of Contextual Embeddings on Graph-Based Dependency Parsing, Koç University, Department of Computer Engineering. May 2020. (PDF, Presentation).
Publications: CoNLL18 and CoNLL17

Thesis Abstract:
I demonstrate the effect of contextual embeddings on transition and graph-based parsing methods and test our contribution, the structured meta biaffine decoder, using various graph-based parsing algorithms.
As Koc University Graph-Based parsing team, we implemented a graph-based dependency parsing model in order to perform syntactic and semantic analysis of given sentences. Our neural graph-based parser consists of two main parts, which are encoder and decoder. The encoder forms continuous feature vectors from provided sentences for the neural graph-based parser to process the texts properly, whereas the decoder produces the parse tree from the output of the neural parser, by first producing a graph representation of the output. We participated in CoNLL 2018 Shared Task with the parsing model we created, and had the opportunity to run our model on 61 different data sets formed with texts in 41 different languages. We took advantage of natural language processing and deep learning techniques, including graph-based dependency parsing algorithms.

Full post...