K-Athena: a performance portable magnetohydrodynamics code

May 27, 2019 by Ingrid Fadelli, Phys.org
An example of the simulations the researchers would like to conduct with K-Athena. The image shows a slice through a 3 dimensional box of a driven turbulence simulation illustrating the fluid/plasma density. Credit: Grete, Glines & O’Shea.

Running large-scale simulations is a crucial aspect of modern scientific research, yet it often requires a vast amount of computational resources. As we approach the era of exascale computing, which will be marked by the introduction of highly performing supercomputers, researchers have been trying to develop new architectures and codes to meet the huge computational requirements of our times. An important property to consider when developing codes for the exascale computing era is performance portability, which prevents the repeated, non-trivial refactoring of a code for different architectures.

To this end, researchers at Michigan State University have recently developed K-Athena, a performance portable magnetohydrodynamic (MHD) . Magnetohydrodynamics (MHD) codes are numerical techniques and algorithms that can help solve problems pertaining electrically conducting fluids. The researchers' code, presented in a paper pre-published on arXiv, combines Athena++, an existing MHD code, with Kokkos, a performance portable on-node parallel programming paradigm.

"In the ongoing process of developing the first exascale supercomputers it became clear that current hardware architectures are unsuitable to reach this goal and that new architectures are required," Philipp Grete, one of the researchers who carried out the study, told TechXplore. "From a software point of view, new architectures usually require a significant rewrite of existing code in order to efficiently make use of new hardware features. The research in our group relies on detailed large-scale simulations and we want to be ready to use the new supercomputers to their full potential the moment they become available."

The underlying objective of the study carried out by Grete and his colleagues was to make codes that they often use in their research performance portable. In other words, the researchers wanted their codes to run at high efficiency on many different architectures, using a single code base.

"Originally, K-Athena was planned as a proof-of-concept combining Kokkos (a performance portability library) with Athena++ (an existing CPU code) to enable GPU-accelerated simulations," Grete explained. "However, over the course of the project and while observing the performance it became clear that K-Athena became more than a proof-of-concept and now is a flexible, production-ready code that runs on any . With this paper, we wanted to share both our experience and the code itself with the broader community."

A further example of the simulations the researchers would like to conduct with K-Athena. The image shows a volume rendering of a decaying Taylor-Green vortex (transition to turbulence problem) illustrating vortical/rotational structures (surfaces) and magnetic field lines (red). Credit: Grete, Glines & O’Shea.

K-Athena works by abstracting the most computationally expensive components of a given simulation. At compile time (i.e. when a source code is translated into machine/hardware code), the code's Kokkos library replaces these abstractions with constructs that are optimal for the specific machine or hardware employed by the user.

"This approach is different from other legacy codes that often maintain multiple code bases to enable simulations on different architectures, such as CPUs or GPUs," Grete said. "Generally speaking, the key advantage of K-Athena that the natural scientist can now focus on the simulation content itself and does not need to be concerned with optimizing code and/or writing multiple version of the source code for different architectures. The latter is 'outsourced' to the computer scientists working on Kokkos together with the hardware vendors."

Tests carried out by Grete and his colleagues suggest that K-Athena's performance portability actually works. In fact, their code ran efficiently on the first, ninth, 24th and 33rd among the fastest supercomputers in the world, spanning four different architectures. The researchers also demonstrated the scalability of their approach by applying it to practically the entire Summit supercomputer (currently the fastest in the world), just about two months after it was officially launched earlier this year.

"From a general point of view, we hope that our findings will encourage other research groups to also explore performance portability in their codes eventually increasing scientific productivity (by spending more time on science and less on rewriting/optimizing code) or to build on the code that we published," Grete said.

The study carried out by Grete and his colleagues offers a valuable example of how code can be adapted to meet the large computational requirements of the modern era. In their paper, the researchers outline their implementation strategies and the challenges they encountered in trying to maximize computational . They hope that their experience and insight will inspire other research teams to start preparing their codes and architecture for the exascale era.

"Our group in generally interested in magnetized turbulence and diffuse plasmas as found in many astrophysical systems," Grete added. "We intend to use K-Athena to simulate those at unprecedented detail both with respect to resolution and the physics we can include. This will allow us to tackle unanswered questions requiring large dynamical scales (e.g., with respect to energy transfers in space plasmas) or related to small scale effects such as magnetic field amplification via the small scale dynamo."

More information: K-Athena: a performance portable structured grid finite volume magnetohydrodynamics code. arXiv:1905.04341 [cs.DC]. arxiv.org/abs/1905.04341

gitlab.com/pgrete/kathena

© 2019 Science X Network