ReliTalk: Relightable Talking Portrait Generation from a Single Video

5 September 2023

Ziwei Liu

ArXiv (abs)PDF HTML Github (124★)

Papers citing "ReliTalk: Relightable Talking Portrait Generation from a Single Video"

34 / 34 papers shown

Title
Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decomposition Jiaxiang Tang Kaisiyuan Wang Hang Zhou Xiaokang Chen Dongliang He Tianshu Hu Jingtuo Liu Gang Zeng Jingdong Wang 3DH 67 80 0 22 Nov 2022
Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation Yu-Ying Yeh Koki Nagano S. Khamis Jan Kautz Xuan Li Ting-Chun Wang 3DH 85 63 0 21 Sep 2022
Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis Shuai Shen Wanhua Li Zhengbiao Zhu Yueqi Duan Jie Zhou Jiwen Lu CVBM 80 107 0 24 Jul 2022
Face Relighting with Geometrically Consistent Shadows Andrew Hou M. Sarkis N. Bi Yiying Tong Xiaoming Liu CVBM 64 41 0 30 Mar 2022
DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Rendering Shunyu Yao Ruizhe Zhong Yichao Yan Guangtao Zhai Xiaokang Yang CVBM 69 92 0 03 Jan 2022
StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation Roy Or-El Xuan Luo Mengyi Shan Eli Shechtman Jeong Joon Park Ira Kemelmacher-Shlizerman 3DH 3DV 114 354 0 21 Dec 2021
I M Avatar: Implicit Morphable Head Avatars from Videos Yufeng Zheng Victoria Fernandez-Abrevaya Marcel C. Bühler Xu Chen Michael J. Black Otmar Hilliges 3DH 111 216 0 14 Dec 2021
FaceFormer: Speech-Driven 3D Facial Animation with Transformers Yingruo Fan Zhaojiang Lin Jun Saito Wenping Wang Taku Komura CVBM 90 204 0 10 Dec 2021
Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields Jonathan T. Barron B. Mildenhall Dor Verbin Pratul P. Srinivasan Peter Hedman 212 1,695 0 23 Nov 2021
FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction Hao Zhu Haotian Yang Longwei Guo Yiding Zhang Yanru Wang Mingkai Huang Menghua Wu Qiu Shen Ruigang Yang Xun Cao CVBM 3DH 103 40 0 01 Nov 2021
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation Hang Zhou Yasheng Sun Wayne Wu Chen Change Loy Xiaogang Wang Ziwei Liu CVBM 113 371 0 22 Apr 2021
MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement Alexander Richard Michael Zollhoefer Yandong Wen Fernando de la Torre Yaser Sheikh CVBM 68 200 0 16 Apr 2021
Towards High Fidelity Face Relighting with Realistic Shadows Andrew Hou Ze Zhang M. Sarkis N. Bi Yiying Tong Xiaoming Liu CVBM 3DH 58 50 0 02 Apr 2021
Neural Video Portrait Relighting in Real-time via Consistency Modeling Longwen Zhang Qixuan Zhang Minye Wu Jingyi Yu Lan Xu 3DH 89 40 0 01 Apr 2021
Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields Jonathan T. Barron B. Mildenhall Matthew Tancik Peter Hedman Ricardo Martín Brualla Pratul P. Srinivasan 123 1,986 0 24 Mar 2021
Learning an Animatable Detailed 3D Face Model from In-The-Wild Images Yao Feng Haiwen Feng Michael J. Black Timo Bolkart CVBM 3DH 112 585 0 07 Dec 2020
NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis Pratul P. Srinivasan Boyang Deng Xiuming Zhang Matthew Tancik B. Mildenhall Jonathan T. Barron 84 605 0 07 Dec 2020
pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis E. R. Chan M. Monteiro Petr Kellnhofer Jiajun Wu Gordon Wetzstein GAN 124 855 0 02 Dec 2020
Shape, Illumination, and Reflectance from Shading Jonathan T. Barron Jitendra Malik 3DV 42 728 0 07 Oct 2020
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild Prajwal K R Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar EGVM 103 790 0 23 Aug 2020
MakeItTalk: Speaker-Aware Talking-Head Animation Yang Zhou Xintong Han Eli Shechtman J. Echevarria E. Kalogerakis Dingzeyu Li 70 423 0 27 Apr 2020
FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction Haotian Yang Hao Zhu Yanru Wang Mingkai Huang Qiu Shen Ruigang Yang Xun Cao CVBM 3DH 86 283 0 31 Mar 2020
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis B. Mildenhall Pratul P. Srinivasan Matthew Tancik Jonathan T. Barron R. Ramamoorthi Ren Ng 129 2,592 0 19 Mar 2020
Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose Ran Yi Zipeng Ye Juyong Zhang Hujun Bao Yong Liu CVBM 84 123 0 24 Feb 2020
Everybody's Talkin': Let Me Talk as You Want Linsen Song Wayne Wu Chao Qian Ran He Chen Change Loy DiffM VGen 79 147 0 15 Jan 2020
Learning Physics-guided Face Relighting under Directional Light Thomas Nestmeyer Jean-François Lalonde Iain Matthews Andreas M. Lehrmann 3DH CVBM 53 121 0 07 Jun 2019
Capture, Learning, and Synthesis of 3D Speaking Styles Daniel Cudeiro Timo Bolkart Cassidy Laidlaw Anurag Ranjan Michael J. Black CVBM 3DH 95 343 0 08 May 2019
Single Image Portrait Relighting Tiancheng Sun Jonathan T. Barron Yun-Ta Tsai Zexiang Xu Xueming Yu Graham Fyffe Christoph Rhemann Jay Busch P. Debevec R. Ramamoorthi 3DH 72 259 0 02 May 2019
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation Hang Zhou Yu Liu Ziwei Liu Ping Luo Xiaogang Wang CVBM 92 443 0 20 Jul 2018
Self-Attention Generative Adversarial Networks Han Zhang Ian Goodfellow Dimitris N. Metaxas Augustus Odena GAN 151 3,732 0 21 May 2018
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric Richard Y. Zhang Phillip Isola Alexei A. Efros Eli Shechtman Oliver Wang EGVM 384 11,920 0 11 Jan 2018
Neural Face Editing with Intrinsic Image Disentangling Zhixin Shu Ersin Yumer Sunil Hadap Kalyan Sunkavalli Eli Shechtman Dimitris Samaras CVBM DRL GAN 100 285 0 13 Apr 2017
Lip Reading Sentences in the Wild Joon Son Chung A. Senior Oriol Vinyals Andrew Zisserman 261 793 0 16 Nov 2016
FaceNet: A Unified Embedding for Face Recognition and Clustering Florian Schroff Dmitry Kalenichenko James Philbin 3DH 389 13,172 0 12 Mar 2015