Lessons Learned from Large Data Visualization Software Development for the K computer

Abstract

High Performance Computing (HPC) always had a close relationship with visualization as we can remember the landmark report on ''Visualization in Scientific Computing'', which was credited to have coined the term Scientific Visualization (SciVis). K computer, a Japanese flagship HPC system, appeared in 2011 as the most powerful supercomputer in the Top500 list, and as other similar HPC systems in that ranking, it was designed to enable ''Grand Challenge'' scientific computing with unprecedented scale and size. RIKEN Center for Computational Science (RIKEN R-CCS) operated and provided the K computer's computational resources to the HPC community for almost 8 years until it was decommissioned in 2019. Considering that most of the scientific computing results were publicly presented in the form of visual images and movies, we can infer that the SciVis was widely applied for assisting the domain scientists with their end-to-end scientific computing workflows. In addition to the traditional visualization applications, various others large data visualization software development were conducted in order to tackle the increased size and amount of the simulation outputs. RIKEN R-CCS participated in some of these development and deployment dealing with several environmental and human factors. Although we have no precise statistics regarding the visualization software usage, in this paper, we would like to present some findings and lessons learned from the large data visualization software development in the K computer environment.