Large-Scale High-Quality 3D Gaussian Head Reconstruction from Multi-View Captures

Problem
This paper addresses the gap in high-quality 3D head reconstruction from multi-view captures, particularly in the context of existing datasets that are limited in scale and diversity. The authors present HeadsUp, a method that leverages a significantly larger internal dataset comprising over 10,000 subjects, which is an order of magnitude larger than current multi-view human head datasets. This work is presented as a preprint and has not yet undergone peer review.

Method
HeadsUp employs a feed-forward encoder-decoder architecture designed to efficiently compress multi-view input images into a compact latent representation. The architecture decouples the number of 3D Gaussians from the number and resolution of input images by utilizing a UV-parameterized representation. This allows the model to generate a set of 3D Gaussians anchored to a neutral head template. The training process involves high-resolution input views, enabling the model to learn robust representations across a diverse set of identities. The authors provide insights into the scaling behavior of the model, analyzing how reconstruction quality varies with the number of identities, input views, and model capacity.

Results
HeadsUp achieves state-of-the-art performance in 3D head reconstruction, outperforming existing methods on benchmark datasets. The authors report significant improvements in reconstruction quality, although specific quantitative metrics (e.g., PSNR, SSIM) against named baselines are not disclosed in the abstract. The model demonstrates strong generalization capabilities, effectively reconstructing novel identities without requiring test-time optimization. The analysis of scaling behavior reveals practical insights into quality-compute trade-offs, suggesting that the model can maintain high fidelity while being computationally efficient.

Limitations
The authors acknowledge that while their method excels in reconstruction quality, it may still be limited by the diversity of the training dataset, which, despite being large, may not encompass all possible head shapes and expressions. Additionally, the reliance on a neutral head template could introduce biases in the representation of non-neutral identities. The paper does not discuss potential limitations related to the generalization of the model to extreme poses or occlusions, which are common challenges in 3D reconstruction tasks.

Why it matters
The implications of this work are significant for downstream applications in computer graphics, virtual reality, and human-computer interaction. The ability to generate high-quality 3D identities and animate them with expression blendshapes opens new avenues for realistic character modeling and interactive experiences. Furthermore, the insights into scaling behavior and quality-compute trade-offs can inform future research in 3D reconstruction and related fields, potentially leading to more efficient and effective methods for capturing and rendering human likenesses.

Authors: Evangelos Ntavelis, Sean Wu, Mohamad Shahbazi, Fabio Maninchedda, Dmitry Kostiaev, Artem Sevastopolsky, Vittorio Megaro, Trevor Phillips et al.
Source: arXiv:2605.04035
URL: https://arxiv.org/abs/2605.04035v1