We leverage finetuned video diffusion models, intrinsic decomposition of videos, and physically-based differentiable rendering to generate high quality materials for 3D models given a text prompt or a single image. We condition a video diffusion model to respect the input geometry and lighting condition. This model produces multiple views of a given 3D model with coherent material properties. Secondly, we use a recent model to extract intrinsics (base color, roughness, metallic) from the generated video. Finally, we use the intrinsics alongside the generated video in a differentiable path tracer to robustly extract PBR materials directly compatible with common content creation tools.
View on arXiv@article{munkberg2025_2506.09665,
title={ VideoMat: Extracting PBR Materials from Video Diffusion Models },
author={ Jacob Munkberg and Zian Wang and Ruofan Liang and Tianchang Shen and Jon Hasselgren },
journal={arXiv preprint arXiv:2506.09665},
year={ 2025 }
}