Detail-driven Geometry Processing Pipeline using Neural Networks

Abstract

Geometry processing is an established field in computer graphics, covering a variety of topics that embody decades-long research. However, with the pressing demand of reality digitization arising in recent years, classic geometry processing solutions are confronted with new challenges.

For almost all geometry processing algorithms, a fundamental requirement is the ability to represent, preserve and reconstruct geometric details. Many established and highly-optimized geometry processing techniques rely heavily on educated user inputs and careful per-instance parameter tuning. However, fueled by the proliferation of consumer-level 3D acquisition devices and growing accessibility of shape modeling applications for ordinary users, there is a tremendous need for automatic geometry processing algorithms that perform robustly even under incomplete and distorted data. In order to transform existing techniques to meet the new requirements, a new mechanism is called for to distill the user expertise in algorithms.

This thesis offers a solution to the aforementioned challenge by utilizing a contem- porary technology from the machine learning community, namely: deep learning. A general geometry processing pipeline includes the following key steps: raw data processing and enhancement, surface reconstruction from raw data, and shape modeling. Over the course of this thesis, we demonstrate how a variety of tasks in each step of the pipeline can be automated and, more importantly, strengthened by incorporating deep learning to leverage consistencies and high-level semantic priors from data.

Specifically, this thesis proposes two point-based geometry processing algorithms that contribute to the raw data processing step, as well as two algorithms involving implicit representations for the surface reconstruction step, and one shape defor- mation algorithm for the last shape modeling step of the geometry processing pipeline. We demonstrate that, by designing suitable deep learning paradigms and integrating them in the existing geometry processing pipeline, we can achieve substantial progress with little or no user guidance especially for challenging, e.g. noise-ridden, undersampled or unaligned, inputs. Correspondingly, the contribu- tions in the thesis aim to enable autonomous and large-scale geometry processing and drive forward the ongoing transition to digitized reality.