CAGLAR GULCEHRE

Research Scientist at DeepMind.
Previously: Applying deep learning to natural language understading, memory, machine translation and optimization. I was a PhD student at MILA group at University of Montreal, advised by Professor Yoshua Bengio.

ca9lar@gmail.com
Google scholar
   

Some of the places that I have worked

DeepMind
Current

Maluuba
Fall 2016

MILA
2012-2017

Deepmind
2008 - 2009

Tubitak
2008 - 2011

METU
2008 - 2009


Recent News

  • Defended my Ph.D. thesis against Christoper Manning (June 2018).
  • Our recent work on Hyperbolic Attention Networks is out: arXiv Link
  • Our paper "Relational inductive biases, deep learning, and graph networks" is on arXiv: arXiv Link
  • Gave a tutorial on Memory Augmented Neural Networks at EMNLP 2017: Tutorial Page
  • Gave a talk at European Council in Luxembourg on Neural Machine Translation and Planning Mechanisms.

Interest Areas

  • Language Understanding and NLP
  • Reinforcement Learning. In particular Hierarchical Reinforcement Learning, Imitation Learning and Model based RL.
  • Learning Representations: Causal, Efficient, Unsupervised, Compositional, Attention
  • Memory: Episodic, Hierarchical
  • Optimization: Meta-learning, Non-convex Optimization
  • Cognitive Science

Selected Publications

Sample Efficient Adaptive Text-to-Speech

Yutian Chen, Yannis Assael, Brendan Shillingford, David Budden, Scott Reed, Heiga Zen, Quan Wang, Luis C. Cobo, Andrew Trask, Ben Laurie, Caglar Gulcehre, Aaron van den Oord, Oriol Vinyals, Nando de Freitas

Pre-print
PDF

Relational inductive biases, deep learning, and graph networks

Peter W. Battaglia, Jessica B. Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, Caglar Gulcehre, Francis Song, Andrew Ballard, Justin Gilmer, George Dahl, Ashish Vaswani, Kelsey Allen, Charles Nash, Victoria Langston, Chris Dyer, Nicholas Hees, Daan Wierstra, Pushmeet Kohli, Matt Botvinick, Oriol Vinyals, Yujia Li, Razvan Pascanu

Pre-print
PDF

Hyperbolic Attention Networks

Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, Nando de Freitas

Pre-print
PDF

Dynamic Neural Turing Machine with Continuous and Discrete Addressing Schemes

Caglar Gulcehre, Sarath Chandar, Kyungyhun Cho, Yoshua Bengio

Neural Computation Journal
PDF

Plan, Attend, Generate: Planning for Sequence to Sequence Models

Francis Dutil+, Caglar Gulcehre+, Adam Trishler, Yoshua Bengio (+ indicates equal contribution)

NIPS 2017
PDF Code (Github)

Memory augmented neural networks with wormhole connections

Caglar Gulcehre, Sarath Chandar, Yoshua Bengio

Pre-print
PDF

Mollifying Networks

Caglar Gulcehre, Marcin Moczulski, Francesco Visin, Yoshua Bengio

ICLR 2017
PDF

Pointing the Unknown Words

Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, Bowen Zhou, Yoshua Begio

ACL 2016
PDF

On integrating a language model into neural machine translation

Caglar Gulcehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, Yoshua Bengio

Elsevier Computer, Speech and Language Journal
PDF

Noisy Activation Functions

Caglar Gulcehre, Marcin Moczulski, Misha Denil, Yoshua Bengio

ICML 2015
PDF

Identifying and attacking the saddle point problem in high-dimensional non-convex optimization

Yann Dauphin, Razvan Pascanu, Caglar Gulcehre, Kyungyhun Cho, Surya Ganguli, Yoshua Bengio

NIPS 2014
PDF

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio

EMNLP 2014
PDF

Knowledge matters: Importance of prior information for optimization

Caglar Gulcehre, Yoshua Bengio

JMLR 2016 (Extended Version), ICLR 2013
PDF

Policy Distillation

Andrei Rusu, Sergio Gomez Colmenarejo, Caglar Gulcehre, Guillaume Desjardins, James Kirkpatrick, Razvan Pascanu, Volodymyr Mnih, Koray Kavukcuoglu, Raia Hadsell

ICLR 2016
PDF

How to construct deep recurrent neural networks

Razvan Pascanu, Caglar Gulcehre, Kyungyhyn Cho, Yoshua Bengio

ICLR 2014
PDF

Gated Orthogonal Recurrent Units: On Learning to Forget

Li Jing+, Caglar Gulcehre+, John Peurifoy, Yichen Shen, Max Tegmark, Marin Soljacic, Yoshua Bengio (+ indicates equal contribution)

Pre-Print
PDF

Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

Junyoung Chung, Caglar Gulcehre, Kyungyhun Cho, Yoshua Bengio

NIPS 2014 (workshop)
PDF