Adaptive Frame Rate Up-conversion with Motion Extraction from 3D Space for 3D Graphic Pipelines

Abstract

A novel 3D Graphic Pipeline for mobile handheld devices is introduced. It s able to achieve increased frame-rate at its output, high picture quality, improved temporal anti-aliasing, thanks to frame-rate upconversion algorithm based on primitive s motion extraction and efficient compensation. Classic rendering process doesn t exploit spatial and temporal coherence of the motion: each frame is rendered by using a lot of the pipeline resources inside geometry, rasteriser and fragment processing stages. Moreover each frame is computed independently from the temporal adjacent one with a brute force approach. Video based algorithms instead (e.g. MPEG type of processing), heavily exploit temporal, spatial and statistical coherency of the content being coded, relying on motion estimation to extract motion vectors from pixel domain during coding process and apply them during motion compensation decoding process. Previous works on 3D graphics were based on the use of impostors, sprite or coherent layers. Those approaches are too much complex and requires high computational power. Other ones based on frame averaging are too much simple: they achieve poor spatial and temporal quality and artifacts become clearly visible as the motion dynamic increases. Proposed approach instead exploits motion temporal coherency by extracting motion vectors from motions of visible primitives in screen space. Temporal coherence has been exploited between key frames using an adaptive motion compensated stage. This method is able to reconstruct the frames without introducing noticeable artifacts on final pictures. Results achieved are 10-15 dB better quality than simple temporal frame average and show a natural motion with no annoying artifacts like real video content shows.