v1v2 (latest)

DiffDub: Person-generic Visual Dubbing Using Inpainting Renderer with Diffusion Auto-encoder

3 November 2023

Chenpeng Du

Kai Yu

Papers citing "DiffDub: Person-generic Visual Dubbing Using Inpainting Renderer with Diffusion Auto-encoder"

29 / 29 papers shown

Title
IM-Portrait: Learning 3D-aware Video Diffusion for Photorealistic Talking Heads from Monocular Videos Yuan Li Ziqian Bai Feitong Tan Zhaopeng Cui S. Fanello Yinda Zhang DiffM VGen 127 0 0 27 Apr 2025
Identity-Preserving Talking Face Generation with Landmark and Appearance Priors Wei‐Tao Zhong Chaowei Fang Yinqi Cai Pengxu Wei Gangming Zhao Liang Lin Guanbin Li 76 79 0 15 May 2023
DAE-Talker: High Fidelity Speech-Driven Talking Face Generation with Diffusion Autoencoder Chenpeng Du Qi Chen Xie Chen K. Yu DiffM 108 51 0 30 Mar 2023
Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expert Jiadong Wang Xinyuan Qian Malu Zhang R. Tan Haizhou Li EGVM 64 100 0 29 Mar 2023
Improving Few-Shot Learning for Talking Face System with TTS Data Augmentation Qi Chen Ziyang Ma Tao Liu Xuejiao Tan Qu Lu Xie Chen K. Yu CVBM 64 5 0 09 Mar 2023
DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Video Zhimeng Zhang Zhipeng Hu W. Deng Changjie Fan Tangjie Lv Yu-qiong Ding 3DH CVBM 103 65 0 07 Mar 2023
Speech Driven Video Editing via an Audio-Conditioned Diffusion Model Dan Bigioi Shubhajit Basak Michał Stypułkowski Maciej Ziȩba H. Jordan R. Mcdonnell Peter Corcoran DiffM VGen 66 36 0 10 Jan 2023
DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation Shuai Shen Wenliang Zhao Zibin Meng Wanhua Li Zhengbiao Zhu Jie Zhou Jiwen Lu DiffM VGen 105 105 0 10 Jan 2023
Diffused Heads: Diffusion Models Beat GANs on Talking-Face Generation Michal Stypulkowski Konstantinos Vougioukas Sen He Maciej Ziȩba Stavros Petridis Maja Pantic DiffM 71 124 0 06 Jan 2023
Deep Learning for Visual Speech Analysis: A Survey Changchong Sheng Gangyao Kuang L. Bai Chen Hou Y. Guo Xin Xu M. Pietikäinen Li Liu VLM 69 35 0 22 May 2022
RePaint: Inpainting using Denoising Diffusion Probabilistic Models Andreas Lugmayr Martin Danelljan Andrés Romero Feng Yu Radu Timofte Luc Van Gool DiffM 355 1,425 0 24 Jan 2022
Towards Realistic Visual Dubbing with Heterogeneous Sources Tianyi Xie Liucheng Liao Cheng Bi Benlai Tang Xiang Yin Jianfei Yang Mingjie Wang Jiali Yao Yang Zhang Zejun Ma VGen 53 41 0 17 Jan 2022
FaceFormer: Speech-Driven 3D Facial Animation with Transformers Yingruo Fan Zhaojiang Lin Jun Saito Wenping Wang Taku Komura CVBM 90 204 0 10 Dec 2021
Diffusion Autoencoders: Toward a Meaningful and Decodable Representation Konpat Preechakul Nattanat Chatthee Suttisak Wizadwongsa Supasorn Suwajanakorn SyDa DiffM 121 434 0 30 Nov 2021
Blended Diffusion for Text-driven Editing of Natural Images Omri Avrahami Dani Lischinski Ohad Fried DiffM 135 954 0 29 Nov 2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units Wei-Ning Hsu Benjamin Bolte Yao-Hung Hubert Tsai Kushal Lakhotia Ruslan Salakhutdinov Abdel-rahman Mohamed SSL 184 3,003 0 14 Jun 2021
Diffusion Models Beat GANs on Image Synthesis Prafulla Dhariwal Alex Nichol 271 7,958 0 11 May 2021
SUPERB: Speech processing Universal PERformance Benchmark Shu-Wen Yang Po-Han Chi Yung-Sung Chuang Cheng-I Jeff Lai Kushal Lakhotia ... Shuyan Dong Shang-Wen Li Shinji Watanabe Abdel-rahman Mohamed Hung-yi Lee SSL 111 943 0 03 May 2021
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation Hang Zhou Yasheng Sun Wayne Wu Chen Change Loy Xiaogang Wang Ziwei Liu CVBM 113 371 0 22 Apr 2021
Denoising Diffusion Implicit Models Jiaming Song Chenlin Meng Stefano Ermon VLM DiffM 295 7,492 0 06 Oct 2020
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild Prajwal K R Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar EGVM 103 790 0 23 Aug 2020
Denoising Diffusion Probabilistic Models Jonathan Ho Ajay Jain Pieter Abbeel DiffM 742 18,364 0 19 Jun 2020
Conformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati James Qin Chung-Cheng Chiu Niki Parmar Yu Zhang ... Wei Han Shibo Wang Zhengdong Zhang Yonghui Wu Ruoming Pang 229 3,160 0 16 May 2020
MakeItTalk: Speaker-Aware Talking-Head Animation Yang Zhou Xintong Han Eli Shechtman J. Echevarria E. Kalogerakis Dingzeyu Li 73 423 0 27 Apr 2020
Towards Automatic Face-to-Face Translation Prajwal K R Rudrabha Mukhopadhyay Jerin Philip Abhishek Jha Vinay P. Namboodiri C. V. Jawahar CVBM 107 175 0 01 Mar 2020
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context Zihang Dai Zhilin Yang Yiming Yang J. Carbonell Quoc V. Le Ruslan Salakhutdinov VLM 260 3,747 0 09 Jan 2019
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric Richard Y. Zhang Phillip Isola Alexei A. Efros Eli Shechtman Oliver Wang EGVM 384 11,920 0 11 Jan 2018
Attention Is All You Need Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan Gomez Lukasz Kaiser Illia Polosukhin 3DV 803 132,454 0 12 Jun 2017
How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks) Adrian Bulat Georgios Tzimiropoulos 3DH CVBM 3DV 106 1,480 0 21 Mar 2017