phoxel

phoxel. A phoxel is a fundamental data structure in computer science and computer graphics, representing a point in a three-dimensional Euclidean space that carries not only positional data but also a comprehensive set of photometric attributes. It is a portmanteau of "photometric" and "voxel," designed to bridge the gap between geometric and appearance modeling for complex real-world scenes. The concept is central to advanced 3D reconstruction and rendering (computer graphics) techniques, particularly for capturing and representing objects with intricate surface properties like translucency, subsurface scattering, and complex BRDFs.

Definition and concept

A phoxel extends the traditional voxel model by encoding a dense set of photometric observations at a discrete spatial location. While a standard voxel in a volumetric dataset might simply represent density or material occupancy, a phoxel stores a high-dimensional vector of visual data. This typically includes information such as RGB values, surface normal vectors, and reflectance parameters captured from multiple viewpoints under varying lighting conditions. The development of the phoxel concept was heavily influenced by work in image-based rendering and light field capture pioneered at institutions like the Massachusetts Institute of Technology and University of California, Berkeley. This data structure is essential for applications requiring photorealistic rendering from arbitrary viewpoints, as it models how light interacts with a material at a fundamental level, moving beyond simple texture mapping.

Comparison to voxels

The primary distinction lies in the semantic content each structure holds. Traditional voxels, used extensively in medical imaging formats like DICOM and in engines like Minecraft, are primarily geometric or volumetric placeholders. In contrast, a phoxel is inherently a photometric entity, designed to solve problems in appearance modeling that pure geometry cannot. For instance, while a polygon mesh augmented with a normal map can approximate detail, it cannot easily model phenomena like the Fresnel effect or anisotropic highlights without complex shader programs. Phoxel-based representations, by storing actual light transport data, enable more accurate synthesis of these effects. This makes them more akin to a dense point cloud used in Lidar scanning, but with vastly richer attached appearance data.

Applications in computer graphics

Phoxel data structures are pivotal in state-of-the-art 3D scanning and virtual reality systems. They form the backbone of many image-based modeling pipelines, where data from arrays of cameras, such as those used in the Light Stage technology developed at the University of Southern California, is consolidated into a phoxel volume. This representation is then used for free-viewpoint television and creating digital assets for the film industry, notably in visual effects studios like Industrial Light & Magic and Weta Digital. Furthermore, phoxel-based approaches are being researched for augmented reality applications to enable realistic real-time rendering of dynamic objects in uncontrolled environments, a challenge for traditional rasterisation pipelines.

Technical implementation

Implementing a phoxel system typically involves a multi-stage computational pipeline. First, data is captured using a multiview stereo rig or a gonioreflectometer, generating thousands of images. These are processed using structure from motion algorithms, often leveraging libraries like OpenCV or COLMAP, to generate a sparse point cloud. This cloud is then regularized into a volumetric grid, where each cell aggregates the photometric data from all images that observe that 3D point. Efficient storage and querying often rely on hierarchical structures like an octree or sparse voxel octree, with research from NVIDIA and Disney Research focusing on GPU acceleration using frameworks like CUDA. The final representation allows for ray tracing or differential rendering to synthesize novel views.

History and development

The conceptual foundations for phoxels were laid in the early 2000s with the convergence of several research threads. Key influences include the Volumetric Visual Hull work from Carnegie Mellon University, the Surface Light Fields research at the Microsoft Research lab, and the Plenoptic function theory. The term itself gained prominence in the 2010s alongside advances in computational photography and the rise of deep learning for neural rendering. Projects like the Digital Emily initiative, a collaboration between the University of Southern California and Image Metrics, demonstrated early practical use. Subsequent evolution has been driven by the demands of the metaverse and autonomous vehicle simulation, requiring ever more accurate digital twins of real-world objects and environments. Category:Computer graphics Category:Data structures Category:3D imaging