Property-Based Testing for Visualization Development

Abstract

As the testing capabilities of current visualization software fail to cover a large space of rendering parameters, we propose to use property-based testing to automatically generate a large set of tests with different parameter sets. By comparing the resulting renderings for pairs of different parameters, we can verify certain effects to be expected in the rendering upon change of a specific parameter. This allows for testing visualization algorithms with a large coverage of rendering parameters. Our proposed approach can also be used in a test-driven manner, meaning the tests can be defined alongside the actual algorithm. Lastly, we show that by integrating the proposed concepts into the existing regression testing pipeline of Inviwo, we can execute the property-based testing process in a continuous integration setup. To demonstrate our approach, we describe use cases where property-based testing can help to find errors during visualization development.