Extending X3D for Distributed Multimedia Processing and Control

Abstract

Web-based applications of interactive 3D computer graphics are showing a tendency to get more interconnected and visually complex. Virtual communities like Second Life demand realism not only in terms of realistic rendering, but also in terms of integrated multimedia content. For these Web-based applications, X3D is the ISO-standard way to specify and manipulate scene descriptions. In terms of multimedia integration, however, X3D offers to specify content only in the form of URLs pointing to files. Modern middleware for distributed multimedia, on the other hand, allows applications to harness the full range of multimedia processing as well as transparent use and full control of both local and remote components. Integrating a full multimedia processing pipeline into X3D would enable Web authors to use, for example, streaming media, post-processing on media streams, or routing between scene elements (e.g., sensors) and elements of multimedia processing (e.g., TV cards). A full integration of multimedia in X3D is yet missing. In this paper, we propose X3D extensions for a seamless mapping of a distributed multimedia flow graph onto an X3D scene graph, making all the features of a distributed multimedia middleware accessible and usable within an X3D scene. We present our proposed specification and implementation of multimedia nodes for X3D. Using examples and implemented X3D application scenarios, we show the simplicity and feasibility of our approach.