3D-ReGen: A Unified 3D Geometry Regeneration Framework

Problem
This paper addresses the gap in the capability of existing 3D generation frameworks, which typically operate in a one-shot manner, converting 2D images or text prompts into 3D objects with limited controllability and fidelity. The authors propose 3D-ReGen, a unified framework that regenerates 3D geometry conditioned on an initial 3D shape, allowing for enhanced control over the output. This work is presented as a preprint and has not yet undergone peer review.

Method
3D-ReGen introduces a novel conditioning mechanism based on VecSet, which enables the model to refine and enhance the input geometry by incorporating fine-grained details. The architecture leverages a self-supervised learning approach, utilizing off-the-shelf 3D datasets to learn a regeneration prior through pretext tasks and data augmentations, thus eliminating the need for additional annotations. The training process involves optimizing a loss function that balances geometric consistency and detail fidelity, although specific loss formulations and training compute details are not disclosed in the paper.

Results
The authors report that 3D-ReGen achieves state-of-the-art performance in several tasks, including 3D enhancement, reconstruction, and editing. Notably, the framework demonstrates significant improvements over baseline methods on benchmarks such as ShapeNet and ModelNet, with quantitative metrics indicating a 15% increase in geometric consistency and a 20% improvement in fine-grained quality compared to leading approaches. These results underscore the effectiveness of the conditioning mechanism and the self-supervised learning strategy employed.

Limitations
The authors acknowledge several limitations, including the reliance on the quality of the initial 3D shape, which can affect the regeneration output. Additionally, the framework may struggle with highly complex geometries or textures that deviate significantly from the training data. The paper does not address potential scalability issues related to the training process or the computational resources required for real-time applications.

Why it matters
The introduction of 3D-ReGen has significant implications for downstream applications in computer graphics, virtual reality, and augmented reality, where controllable and high-fidelity 3D object generation is crucial. By enabling fine-grained editing and enhancement of 3D shapes, this framework could facilitate more interactive design processes and improve the realism of generated content. Furthermore, the self-supervised learning approach may inspire future research into unsupervised methods for 3D shape generation, potentially broadening the accessibility of high-quality 3D modeling tools.

Authors: Geon Yeong Park, Roman Shapovalov, Rakesh Ranjan, Jong Chul Ye, Andrea Vedaldi, Thu Nguyen-Phuoc
Source: arXiv:2604.28134
URL: https://arxiv.org/abs/2604.28134v1