Text-to-3D Generation

思路：

diffusion基于文本生成二维图像成功的原因：可以从网络上收集到大量的图片文本对。

三维生成大模型的难点：三维数据稀缺，目前只能利用二维的生成能力提升迁移到三维

1. 端到端，直接生成三维数据
   • 生成mesh：图卷积网络
   • 生成点云：point-E
2. 利用diffusion，从二维升三维
   • text-image，然后利用多视角图像进行三维重建，要求diffusion对多视角一致性
   • 利用多视角diffusion的prior，然后使用分数蒸馏采样SDS损失

• Magic3d：High-resolution text-to-3d content creation

• Dreambooth3d: Subject-driven text-to-3d generation

• Fantasia3d: Disentangling geometry and appearance for high-quality textto-3d content creation

• Prolificdreamer: High-fidelity and diverse textto-3d generation with variational score distillation.

• Dreamgaussian: Generative gaussian splatting for efficient 3d content creation

Layout

1. Compositional 3d scene generation using locally conditioned diffusion
2. Componerf: Text-guided multi-object compositional nerf with editable 3d scene layout.
3. Set-the-scene: Global-local training for generating controllable nerf scenes.
4. LOOSEControl
5. GALA3D