Model-Agnostic Visual Explanation of Machine Learning Models Based on Heat Map

Abstract

It is essential to assess the trustworthiness of the machine learning models when deploying them to real-world applications, such as healthcare and risk management, in which domain experts need to make critical decisions. We propose a visual analysis method for supporting domain experts to understand and improve a given machine learning model based on a model-agnostic interpretable explanation technique. Our visualization method provides a heat map matrix as an overview of the model explanation and helps efficient feature engineering and data cleaning. We demonstrate our visualization method on a text classification task.