Deep Learning Seminar  /  February 13, 2020

An Introduction to Natural Language Processing (Part II)

Abstract:

Continuing from last weeks lecture, in this talk I will focus on the task of machine translation.
I will start with the original encoder-decoder architectures, highlighting in particular their weaknesses. This will motivate the mechanism of attention and the construction of transformer networks. Lastly, I will present the BERT architecture and the state-of-the-art reformer networks.

  • Technomathematics (Dep. of Mathematics)
  • Mechatronics in Mechenical and Automotive Engineering (Dep. of Mechanical and Process Engineering)
  • Automation Control (Dep. of Electrical and Computer Engineering)
  • Electromobility (Dep. of Electrical and Computer Engineering)

The seminar takes place at the ITWM every 1st Tuesday of a month (besides holidays and summer break). Aims are broadening of experiences and exchange of scientific views – also beyond the organizing groups.

Typical subjects of talks are:

  • ongoing or recently finished graduations and doctoral theses
  • current research and projects

The topics vary from mathematical methods to technical implementations. Usually, the talks present research results. However, some show open issues for brainstorming and inputs from the audience.

The seminar »KL-Regelungstechnik« (Kaiserslautern – Control Theory and Control Engineering) is organized by our department as well as several research groups of the TU Kaiserslautern:

  • Technomathematics (Dep. of Mathematics)
  • Mechatronics in Mechenical and Automotive Engineering (Dep. of Mechanical and Process Engineering)
  • Automation Control (Dep. of Electrical and Computer Engineering)
  • Electromobility (Dep. of Electrical and Computer Engineering)

The seminar takes place at the ITWM every 1st Tuesday of a month (besides holidays and summer break). Aims are broadening of experiences and exchange of scientific views – also beyond the organizing groups.

Typical subjects of talks are:

  • ongoing or recently finished graduations and doctoral theses
  • current research and projects

The topics vary from mathematical methods to technical implementations. Usually, the talks present research results. However, some show open issues for brainstorming and inputs from the audience.

The seminar »KL-Regelungstechnik« (Kaiserslautern – Control Theory and Control Engineering) is organized by our department as well as several research groups of the TU Kaiserslautern:

  • Technomathematics (Dep. of Mathematics)
  • Mechatronics in Mechenical and Automotive Engineering (Dep. of Mechanical and Process Engineering)
  • Automation Control (Dep. of Electrical and Computer Engineering)
  • Electromobility (Dep. of Electrical and Computer Engineering)

The seminar takes place at the ITWM every 1st Tuesday of a month (besides holidays and summer break). Aims are broadening of experiences and exchange of scientific views – also beyond the organizing groups.

Typical subjects of talks are:

  • ongoing or recently finished graduations and doctoral theses
  • current research and projects

The topics vary from mathematical methods to technical implementations. Usually, the talks present research results. However, some show open issues for brainstorming and inputs from the audience.