MODELAR: A MODular and EvaLuative framework to improve surgical Augmented Reality visualization

Abstract

The use of Augmented Reality (AR) for the visualization of 3D biomedical image data is possible thanks to a growing number of hardware and software solutions. Considerable efforts are made during surgery, where the visual information of the target structures can either be highlighted or dulled. However, as technical challenges and barriers to development decrease, it's increasingly important to take into account the specific capacities and constraints of the surgeon's perceptual and cognitive systems. To address this legitimate problem, we present a practical framework that evaluates the importance of visual encodings and renderings for surgical AR. By conducting a task-specific user study we observed a set of emerging visualization strategies. The given task is to make the kidney boundary visually salient and make the tumor and calyx distinguishable. After having recruited 23 participants, we found two preferred presets to tackle this task. With both presets, the usage of color, depth, and opacity improved the display of the organ bounds while contrasting the tumor and calyx. 19 participants successfully completed the task using MODELAR. Their preference was to either find a good preset where the organ bounds were visible then adjust the color of target objects or vice versa. MODELAR helped us better identify effective visualization that best fit the task requirements. Our evaluation results and the modular framework MODELAR is freely available and open source at https://github.com/ghattab/MODELAR.