计算机科学
人工智能
计算机视觉
渲染(计算机图形)
光辉
忠诚
卷积神经网络
计算机图形学(图像)
电信
物理
光学
作者
Ajay N. Jain,Ben Mildenhall,Jonathan T. Barron,Pieter Abbeel,Ben Poole
标识
DOI:10.1109/cvpr52688.2022.00094
摘要
We combine neural rendering with multi-modal image and text representations to synthesize diverse 3D objects solely from natural language descriptions. Our method, Dream Fields, can generate the geometry and color of a wide range of objects without 3D supervision. Due to the scarcity of diverse, captioned 3D data, prior methods only generate objectsfrom a handful of categories, such as ShapeNet. Instead, we guide generation with image-text models pre-trained on large datasets of captioned images from the web. Our method optimizes a Neural Radiance Field from many camera views so that rendered images score highly with a target caption according to a pre-trained CLIP model. To improve fidelity and visual quality, we introduce simple geometric priors, including sparsity-inducing transmittance regularization, scene bounds, and new MLP architectures. In experiments, Dream Fields produce realistic, multi-view consistent object geometry and color from a variety of natural language captions.
科研通智能强力驱动
Strongly Powered by AbleSci AI