15 Rethinking 3D Object Reconstruction When Inputs Are Incomplete

Standard 3D generation research relies on ideal conditions, such as single clean objects with complete visibility. Real-world data is messy, occluded, and sparse, creating a gap between academic benchmarks and practical deployment.

Axolotl3D bridges this gap by recognizing that shape reconstruction, scene extraction, and interactive editing all share the same underlying problem: completing a full shape from partial observations. Instead of relying on separate narrow tools, it fuses multimodal inputs into tokens that condition a diffusion transformer, generating complete 3D shape latents across varied scenarios.

• Real-world 3D captures usually contain gaps, occlusion, and missing viewpoints.
• Fusing input modalities into unified tokens allows a single diffusion transformer to handle multiple tasks.
• Training with dynamically mixed partial inputs builds cross-modal reasoning resilience.
• Interactive 2D painting can be lifted into 3D while holding unedited point cloud regions fixed.
• Raw Gaussian splatting often leaves holes, whereas completed meshes support true physics simulation.

Future iterations of this framework could balance strict adherence to input points with greater tolerance for noisy sensor data.

What new workflows become possible when 3D editing and shape completion are combined into a single unified model?