PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual World

Problem
This paper addresses the gap in the generation of physics-grounded 3D assets for interactive virtual environments, particularly for applications in embodied AI. Existing methodologies primarily focus on static geometric representations, neglecting the functional properties necessary for realistic interactions. The authors propose that effective asset generation should be based on a comprehensive understanding of functional logic and hierarchical physics. This work is presented as a preprint and has not yet undergone peer review.

Method
The authors introduce PhysForge, a decoupled two-stage framework designed to generate interactive 3D assets. The first stage employs a Vision-Language Model (VLM) that acts as a “physical architect,” creating a “Hierarchical Physical Blueprint.” This blueprint encompasses material properties, functional requirements, and kinematic constraints. The second stage utilizes a physics-grounded diffusion model to synthesize high-fidelity geometry and accurate kinematic parameters. A novel KineVoxel Injection (KVI) mechanism is introduced to enhance the integration of kinematic data into the generated assets. The framework is supported by PhysDB, a large-scale dataset comprising 150,000 assets annotated with four tiers of physical properties, which facilitates the training and evaluation of the model.

Results
PhysForge demonstrates significant improvements in generating functionally plausible and simulation-ready 3D assets compared to existing baselines. The authors report that their method outperforms traditional asset generation techniques on various benchmarks, although specific numerical results and effect sizes are not disclosed in the abstract. The qualitative assessments indicate that the generated assets exhibit enhanced realism and interactivity, making them suitable for use in interactive virtual worlds and embodied AI applications.

Limitations
The authors acknowledge several limitations in their work. Firstly, the reliance on a large-scale dataset may introduce biases based on the diversity and quality of the assets included in PhysDB. Additionally, the complexity of the KVI mechanism may limit its scalability to more diverse asset types. The authors also note that while the framework is designed for interactive applications, the evaluation metrics primarily focus on visual fidelity and functional plausibility, potentially overlooking other critical aspects such as computational efficiency during runtime. Furthermore, the paper does not address the potential challenges in real-time asset generation, which is crucial for interactive environments.

Why it matters
The implications of this work are significant for the fields of computer graphics, game development, and embodied AI. By providing a robust framework for generating physics-grounded 3D assets, PhysForge can enhance the realism and interactivity of virtual environments, thereby improving user experience and engagement. The introduction of a hierarchical approach to asset generation may also inspire future research into more sophisticated models that integrate functional logic with physical realism. This work lays the groundwork for further exploration into automated content creation, potentially reducing the time and resources required for asset development in interactive applications.

Authors: Yunhan Yang, Chunshi Wang, Junliang Ye, Yang Li, Zanxin Chen, Zehuan Huang, Yao Mu, Zhuo Chen et al.
Source: arXiv:2605.05163
URL: https://arxiv.org/abs/2605.05163v1