Scalable Parallel Programming

Together with 13 European partners we are involved in the project ExaNoDe (European Exascale Processor Memory Node Design). This project will investigate, develop, integrate and pilot the building blocks for a highly efficient, highly integrated, multi-way, high-performance, heterogeneous compute element aimed towards Exascale computing. 

We develop the Fraunhofer GASPI/GPI, which is an open-source communication library to be used for communication between computing nodes. The API has been designed with a variety of possible memory spaces in mind.  GASPI/GPI provides configurable memory segments which aim at mapping the hardware configuration and making them available for the application.

-> More on ExaNoDe

This project addresses the problem of programming model design and implementation for the Exascale. The first Exascale computers will be very highly parallel systems, consisting of a hierarchy of architectural levels. To program such systems effectively and portably, programming APIs with efficient and robust implementations must be ready in the appropriate timescale.

We introduce the programming model GASPI and its implementation GPI into the project and evaluate the interoperability requirements with a number of applications such as the Computational Fluid Dynamics Code TAU of DLR (German Aerospace) for aerodynamic simulation.

-> More on INTERWinE

The EPiGRAM project which concluded 2016 has worked on programming models for exascale systems. Exascale computing power is likely to be reached in the next decade. While the precise system architectures are still evolving, one can safely assume that they will be largely based on deep hierarchies of multicore CPUs with similarly deep memory hierarchies, and likely also supported by accelerators.

Appropriate programming models are needed to allow applications to run efficiently at large scale on these platforms. Message Passing (MPI) has emerged as the de-facto standard for parallel programming on current Petascale machines; but Partitioned Global Address Space (PGAS) languages and libraries are increasingly being considered as alternatives or complements to MPI. These models will likely also play an important role in Exascale systems. However, both approaches have problems that will prevent them reaching Exascale performance.  In the EPiGRAM project, we addressed some of the main limitations of MP and PGAS programming models by: investigating new disruptive concepts and algorithms in Exascale programming models; providing prototypical implementations; and validating our approach with three real-world applications that have the potential for reaching Exascale performance.

The GASPI programming model with its GPI implementation, which is developed and maintained by Fraunhofer ITWM, is a PGAS representative.  EPiGRAM made GASPI and GPI-2 more complete and robust by closing gaps preventing GASPI and GPI-2 to go for Exascale. Within EPiGRAM strong scaling of the RTM application up to 60k cores at SuperMUC have been shown to be almost linear.

The EXA2CT project which concluded in 2016 brought together experts at the cutting edge of the development of solvers, related algorithmic techniques, and HPC software architects for programming models and communication.

In the scope of the project modular open source proto-applications were developed that demonstrate the algorithms and programming techniques developed in the project, to help boot-strap the creation of genuine Exascale codes. Technologies developed in EXA2CT range from advanced libraries such as ExaShark and GASPI/GPI that help to program massively parallel machines to solver algorithms improved by better overlapping communication and computation and by increasing the arithmetic intensity. All of this is verified on industry relevant prototype applications.

Tom van der Aa (ExaScience Lifelab at imec, Belgium), coordinator of the EXA2CT project states that In the project it could be shown that Fraunhofer ITWM’s programming API GPI-2 can genuinely outperform MPI.