Research Talk by Sebastian Ruder: Cross-lingual and continual transfer learning in NLP

Conversation is key to innovation. At Google Berlin, we are organizing this Research Talk Series to share our work and to engage with the local research and engineering community. The talks provide the opportunity to find out more about the challenging problems we are working on, approaches, utilized techniques, results, applications in products, fundamental research as well as research developments and innovation across a range of computer science disciplines. We invite researchers and engineers from academia and industry to this talk. Join us for an opportunity to connect with members of Google’s engineering team, as well as your peers from the region.

Registration starts at 5:30 p.m. and the talk will begin at 6:00 p.m.. Food and drinks will be provided before and after the talk.

Please confirm your attendance on the guest list by REGISTERING BY 10 July 2019. Please note that the number of seats is limited.

Cross-lingual and continual transfer learning in NLP

As state-of-the-art models in natural language processing have become more powerful, two key challenges are a) to scale them to other languages and b) to apply them to other tasks without forgetting acquired knowledge. In this talk, I will analyze the limitations of unsupervised cross-lingual word embedding models, which have become popular over the last year. Such approaches learn representations without any form of supervision, but still fail in challenging conditions. Furthermore, I will discuss how episodic memory can be used to retain previously learned information in continual learning in NLP. Episodic memory is commonly used in reinforcement learning to store and replay past experience, but can be a key factor in preventing catastrophic forgetting.

We look forward to seeing you!

Benjamin, Jakob & The Google Berlin Team

Email to bhuetteroth@google.com

Agenda