A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild

23 August 2020

Prajwal K R

Rudrabha Mukhopadhyay

Papers citing "A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild"

50 / 410 papers shown

Title
Diffused Heads: Diffusion Models Beat GANs on Talking-Face Generation Michal Stypulkowski Konstantinos Vougioukas Sen He Maciej Ziȩba Stavros Petridis M. Pantic DiffM 24 116 0 06 Jan 2023
CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior Jinbo Xing Menghan Xia Yuechen Zhang Xiaodong Cun Jue Wang T. Wong 32 141 0 06 Jan 2023
Self-Supervised Video Forensics by Audio-Visual Anomaly Detection Chao Feng Ziyang Chen Andrew Owens 31 71 0 04 Jan 2023
StyleTalk: One-shot Talking Head Generation with Controllable Speaking Styles Yifeng Ma Suzhe Wang Zhipeng Hu Changjie Fan Tangjie Lv Yu-qiong Ding Zhidong Deng Xin Yu 58 82 0 03 Jan 2023
How Do Deepfakes Move? Motion Magnification for Deepfake Source Detection U. Ciftci Ilke Demir 30 6 0 28 Dec 2022
Memories are One-to-Many Mapping Alleviators in Talking Face Generation Anni Tang Tianyu He Xuejiao Tan Jun Ling Liang Song CVBM 26 23 0 09 Dec 2022
Masked Lip-Sync Prediction by Audio-Visual Contextual Exploitation in Transformers Yasheng Sun Hang Zhou Kaisiyuan Wang Qianyi Wu Zhibin Hong Jingtuo Liu Errui Ding Jingdong Wang Ziwei Liu Koike Hideki 35 34 0 09 Dec 2022
Learning to Dub Movies via Hierarchical Prosody Models Gaoxiang Cong Liang Li Yuankai Qi Zhengjun Zha Qi Wu Wen-yu Wang Bin Jiang Ming Yang Qin Huang 75 25 0 08 Dec 2022
Talking Head Generation with Probabilistic Audio-to-Visual Diffusion Priors Zhentao Yu Zixin Yin Deyu Zhou Duomin Wang Finn Wong Baoyuan Wang DiffM 30 35 0 07 Dec 2022
Audio-Driven Co-Speech Gesture Video Generation Xian Liu Qianyi Wu Hang Zhou Yuanqi Du Wayne Wu Dahua Lin Ziwei Liu SLR VGen 36 47 0 05 Dec 2022
FakeOut: Leveraging Out-of-domain Self-supervision for Multi-modal Video Deepfake Detection Gil Knafo Ohad Fried 23 5 0 01 Dec 2022
High-fidelity Facial Avatar Reconstruction from Monocular Video with Generative Priors Yun-Hao Bai Yanbo Fan Xuanxia Wang Yong Zhang Jingxiang Sun Chun Yuan Ying Shan 3DH 29 25 0 28 Nov 2022
VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild K. Cheng Xiaodong Cun Yong Zhang Menghan Xia Fei Yin Mingrui Zhu Xuanxia Wang Jue Wang Nan Wang CVBM 25 92 0 27 Nov 2022
Progressive Disentangled Representation Learning for Fine-Grained Controllable Talking Head Synthesis Duomin Wang Yu Deng Zixin Yin H. Shum Baoyuan Wang 16 60 0 26 Nov 2022
Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decomposition Jiaxiang Tang Kaisiyuan Wang Hang Zhou Xiaokang Chen Dongliang He Tianshu Hu Jingtuo Liu Gang Zeng Jingdong Wang 3DH 40 76 0 22 Nov 2022
SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation Wenxuan Zhang Xiaodong Cun Xuan Wang Yong Zhang Xiaodong Shen Yu-Xiao Guo Ying Shan Fei Wang VGen 35 233 0 22 Nov 2022
SPACE: Speech-driven Portrait Animation with Controllable Expression Francesco Ferroni Arun Mallya Ting-Chun Wang Rafael Valle Xuan Li VGen 34 45 0 17 Nov 2022
MARLIN: Masked Autoencoder for facial video Representation LearnINg Zhixi Cai Shreya Ghosh Kalin Stefanov Abhinav Dhall Jianfei Cai Hamid Rezatofighi Reza Haffari Munawar Hayat ViT CVBM 27 60 0 12 Nov 2022
SyncTalkFace: Talking Face Generation with Precise Lip-Syncing via Audio-Lip Memory Se Jin Park Minsu Kim Joanna Hong J. Choi Y. Ro CVBM 30 85 0 02 Nov 2022
Multimodal Transformer Distillation for Audio-Visual Synchronization Xuan-Bo Chen Haibin Wu Chung-Che Wang Hung-yi Lee J. Jang 26 3 0 27 Oct 2022
Pre-Avatar: An Automatic Presentation Generation Framework Leveraging Talking Avatar Aolan Sun Xulong Zhang Tiandong Ling Jianzong Wang Ning Cheng Jing Xiao 35 4 0 13 Oct 2022
Compressing Video Calls using Synthetic Talking Heads Madhav Agarwal Anchit Gupta Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar 17 10 0 07 Oct 2022
A Keypoint Based Enhancement Method for Audio Driven Free View Talking Head Synthesis Yichen Han Ya Li Yingming Gao Jinlong Xue Songpo Wang Lei Yang 21 2 0 07 Oct 2022
Audio-Visual Face Reenactment Madhav Agarwal Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar DiffM VGen 27 22 0 06 Oct 2022
AutoLV: Automatic Lecture Video Generator Wen Wang Yang Song Sanjay Jha VGen 16 3 0 19 Sep 2022
Continuously Controllable Facial Expression Editing in Talking Face Videos Zhiyao Sun Yu-Hui Wen Tian Lv Y. Sun Ziyang Zhang Yaoyuan Wang Yong-Jin Liu CVBM 37 10 0 17 Sep 2022
Talking Head from Speech Audio using a Pre-trained Image Generator M. M. Alghamdi He Wang A. Bulpitt David C. Hogg 75 21 0 09 Sep 2022
Facial Expression Translation using Landmark Guided GANs Hao Tang N. Sebe CVBM 24 3 0 05 Sep 2022
Synthesizing Photorealistic Virtual Humans Through Cross-modal Disentanglement S. Ravichandran Ondrej Texler Dimitar Dinev Hyun Jae Kang 17 4 0 03 Sep 2022
Lip-to-Speech Synthesis for Arbitrary Speakers in the Wild Sindhu B. Hegde Prajwal K R Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar 45 10 0 01 Sep 2022
StableFace: Analyzing and Improving Motion Stability for Talking Face Generation Jun Ling Xuejiao Tan Liyang Chen Runnan Li Yuchao Zhang Sheng Zhao Liang Song CVBM 47 13 0 29 Aug 2022
StyleTalker: One-shot Style-based Audio-driven Talking Head Video Generation Dong Min Min-Hwan Song Eunji Ko Sung Ju Hwang VGen 38 12 0 23 Aug 2022
DualVoice: Speech Interaction that Discriminates between Normal and Whispered Voice Input Jun Rekimoto 22 6 0 22 Aug 2022
Towards MOOCs for Lipreading: Using Synthetic Talking Heads to Train Humans in Lipreading at Scale Aditya Agarwal Bipasha Sen Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar 25 0 0 21 Aug 2022
FaceOff: A Video-to-Video Face Swapping System Aditya Agarwal Bipasha Sen Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar PICV CVBM 45 2 0 21 Aug 2022
Extreme-scale Talking-Face Video Upsampling with Audio-Visual Priors Sindhu B. Hegde Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar CVBM 16 1 0 17 Aug 2022
Study of detecting behavioral signatures within DeepFake videos Qiaomu Miao Sinhwa Kang Stacy Marsella S. DiPaola Chao Wang Ari Shapiro 22 3 0 06 Aug 2022
Video Manipulations Beyond Faces: A Dataset with Human-Machine Analysis Trisha Mittal Ritwik Sinha Viswanathan Swaminathan John Collomosse Tianyi Zhou 30 9 0 26 Jul 2022
Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis Shuai Shen Wanhua Li Zhengbiao Zhu Yueqi Duan Jie Zhou Jiwen Lu CVBM 27 105 0 24 Jul 2022
Audio-driven Neural Gesture Reenactment with Video Motion Graphs Yang Zhou Jimei Yang Dingzeyu Li Jun Saito Deepali Aneja E. Kalogerakis DiffM SLR 42 20 0 23 Jul 2022
Object-Compositional Neural Implicit Surfaces Qianyi Wu Xian Liu Yuedong Chen Kejie Li Chuanxia Zheng Jianfei Cai Jianmin Zheng OCL AI4CE 38 74 0 20 Jul 2022
Cut Inner Layers: A Structured Pruning Strategy for Efficient U-Net GANs Bo-Kyeong Kim Shinkook Choi Hancheol Park 21 4 0 29 Jun 2022
Face-Dubbing++: Lip-Synchronous, Voice Preserving Translation of Videos Alexander Waibel M. Behr Fevziye Irem Eyiokur Dogucan Yaman Tuan-Nam Nguyen Carlos Mullov Mehmet Arif Demirtas Alperen Kantarci Stefan Constantin H. K. Ekenel CVBM 15 14 0 09 Jun 2022
EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model Xinya Ji Hang Zhou Kaisiyuan Wang Qianyi Wu Wayne Wu Feng Xu Xun Cao CVBM 57 157 0 30 May 2022
Merkel Podcast Corpus: A Multimodal Dataset Compiled from 16 Years of Angela Merkel's Weekly Video Podcasts Debjoy Saha Shravan Nayak Timo Baumann 24 3 0 24 May 2022
Deep Learning for Visual Speech Analysis: A Survey Changchong Sheng Gangyao Kuang L. Bai Chen Hou Y. Guo Xin Xu M. Pietikäinen Li Liu VLM 29 33 0 22 May 2022
Talking Face Generation with Multilingual TTS Hyoung-Kyu Song Sanghyun Woo Junhyeok Lee S. Yang Hyunjae Cho Youseong Lee Dongho Choi Kang-Wook Kim CVBM 40 21 0 13 May 2022
Deep Learning and Synthetic Media Raphaël Millière 23 18 0 11 May 2022
Learning Visual Styles from Audio-Visual Associations Tingle Li Yichen Liu Andrew Owens Hang Zhao DiffM 23 21 0 10 May 2022
Emotion-Controllable Generalized Talking Face Generation Sanjana Sinha S. Biswas Ravindra Yadav Brojeshwar Bhowmick CVBM 18 49 0 02 May 2022