Enabling Dependability-Driven Resource Use and Message Log-Analysis for Cluster System Diagnosis

Thuan Chuah, Arshad Jhumka, Samantha Alt, Theodoros Damoulas, Nentawe Gurumdimma, Marie-Christine Sawley, Bill Barth, Tommy Minyard, James Browne

Research output: Chapter in Book/Report/Conference proceedingPublished conference contribution

8 Citations (Scopus)

Abstract

Recent work have used both failure logs and resource use data separately (and together) to detect system failure-inducing errors and to diagnose system failures. System failure occurs as a result of error propagation and the (unsuccessful) execution of error recovery mechanisms. Knowledge of error propagation patterns and unsuccessful error recovery is important for more accurate and detailed failure diagnosis, and knowledge of recovery protocols deployment is important for improving system reliability. This paper presents the CORRMEXT framework which carries failure diagnosis another significant step forward by analyzing and reporting error propagation patterns and degrees of success and failure of error recovery protocols. CORRMEXT uses both error messages and resource use data in its analyses. Application of CORRMEXT to data from the Ranger supercomputer have produced new insights. CORRMEXT has: (i) identified correlations between resource use counters that capture recovery attempts after an error, (ii) identified correlations between error events to capture error propagation patterns within the system, (iii) identified error propagation and recovery paths during system execution to explain system behaviour, (iv) showed that the earliest times of change in system behaviour can only be identified by analyzing both the correlated resource use counters and correlated errors. CORRMEXT will be installed on the HPC clusters at the Texas Advanced Computing Center in Autumn 2017.
Original languageEnglish
Title of host publication2017 IEEE 24th International Conference on High Performance Computing (HiPC)
PublisherIEEE Explore
DOIs
Publication statusPublished - 21 Dec 2017

Fingerprint

Dive into the research topics of 'Enabling Dependability-Driven Resource Use and Message Log-Analysis for Cluster System Diagnosis'. Together they form a unique fingerprint.

Cite this