Deep Learning Seminar  /  October 10, 2019

Advanced Deep Learning Acceleration Strategies

An Overview on Recent Papers about Accelerating DL Workloads

Abstract:

The basic steps for a practitioner of Deep Learning (DL) is to use a proper DL framework in combination with a solid computational kernel library, such as Nvidia's cuDNN or Intel's MKL. The major frameworks provide the means to set up a fast and efficient training pipeline, which is likely enough to satisfy most users.

However, as more complex problems in DL are being analyzed and more elaborate training procedures emerge, there is a need to further develope the training environment to enable that kind of research. Users are primarly interested in speed, as it determines how fast they can progress with their research and developement.

This overview presents some papers about recent research on how to accelerate DL training. Rather than giving a broad overview, this presentation will look at a view papers in more detail. The topics will cover data loading, parallelism on GPU as well as CPU, and distributed learning strategies.

The seminar »KL-Regelungstechnik« (Kaiserslautern – Control Theory and Control Engineering) is organized by our department as well as several research groups of the TU Kaiserslautern:

  • Technomathematics (Dep. of Mathematics)
  • Mechatronics in Mechenical and Automotive Engineering (Dep. of Mechanical and Process Engineering)
  • Automation Control (Dep. of Electrical and Computer Engineering)
  • Electromobility (Dep. of Electrical and Computer Engineering)

The seminar takes place at the ITWM every 1st Tuesday of a month (besides holidays and summer break). Aims are broadening of experiences and exchange of scientific views – also beyond the organizing groups.

Typical subjects of talks are:

  • ongoing or recently finished graduations and doctoral theses
  • current research and projects

The topics vary from mathematical methods to technical implementations. Usually, the talks present research results. However, some show open issues for brainstorming and inputs from the audience.

The seminar »KL-Regelungstechnik« (Kaiserslautern – Control Theory and Control Engineering) is organized by our department as well as several research groups of the TU Kaiserslautern:

  • Technomathematics (Dep. of Mathematics)
  • Mechatronics in Mechenical and Automotive Engineering (Dep. of Mechanical and Process Engineering)
  • Automation Control (Dep. of Electrical and Computer Engineering)
  • Electromobility (Dep. of Electrical and Computer Engineering)

The seminar takes place at the ITWM every 1st Tuesday of a month (besides holidays and summer break). Aims are broadening of experiences and exchange of scientific views – also beyond the organizing groups.

Typical subjects of talks are:

  • ongoing or recently finished graduations and doctoral theses
  • current research and projects

The topics vary from mathematical methods to technical implementations. Usually, the talks present research results. However, some show open issues for brainstorming and inputs from the audience.

The seminar »KL-Regelungstechnik« (Kaiserslautern – Control Theory and Control Engineering) is organized by our department as well as several research groups of the TU Kaiserslautern:

  • Technomathematics (Dep. of Mathematics)
  • Mechatronics in Mechenical and Automotive Engineering (Dep. of Mechanical and Process Engineering)
  • Automation Control (Dep. of Electrical and Computer Engineering)
  • Electromobility (Dep. of Electrical and Computer Engineering)

The seminar takes place at the ITWM every 1st Tuesday of a month (besides holidays and summer break). Aims are broadening of experiences and exchange of scientific views – also beyond the organizing groups.

Typical subjects of talks are:

  • ongoing or recently finished graduations and doctoral theses
  • current research and projects

The topics vary from mathematical methods to technical implementations. Usually, the talks present research results. However, some show open issues for brainstorming and inputs from the audience.