A Generic Model for Projection Alignment Applied to Neural Network Visualization

Abstract

Dimensionality reduction techniques are popular tools for the visualization of neural network models due to their ability to display hidden layer activations and aiding the understanding of how abstract representations are being formed. However, many techniques render poor results when used to compare multiple projections resulted from different feature sets, such as the outputs of different hidden layers or the outputs from different models processing the same data. This problem occurs due to the lack of an alignment factor to ensure that visual differences represent actual differences between the feature sets and not artifacts generated by the technique. In this paper, we propose a generic model to align multiple projections when visualizing different feature sets that can be applied to any gradient descent-based dimensionality reduction technique. We employ this model to generate a variant of the UMAP method and show the results of its application.