Reflections on the Developments of Visual Analytics Systems for the K Computer System Log Data

Abstract

Flagship-class high-performance computing (HPC) systems, also known as supercomputers, are large, complex systems that require particular attention for continuous and long-term stable operations. The K computer was a Japanese flagship-class supercomputer ranked as the fastest supercomputer in the Top500 ranking when it first appeared. It was composed of more than eighty thousand compute nodes and consumed more than 12 MW when running the LINPACK benchmark for the Top500 submission. A combined power substation, with a natural gas co-generation system (CGS), was used for the power supply, and also a large air/water cooling facility was used to extract the massive heat generated from this HPC system. During the years of its regular operation, a large log dataset has been generated from the K computer system and its facility, and several visual analytics systems have been developed to better understand the K computer's behavior during the operation as well as the probable correlation of operational temperature with the critical hardware failures. In this paper, we will reflect on these visual analytics systems, mainly developed by graduate students, intended to be used by different types of end users on the HPC site. In addition, we will discuss the importance of collaborative development involving the end users, and also the importance of technical people in the middle for assisting in the deployment and possible continuation of the developed systems.