A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild

23 August 2020

Prajwal K R

Rudrabha Mukhopadhyay

Papers citing "A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild"

50 / 410 papers shown

Title
Talking Head Generation Driven by Speech-Related Facial Action Units and Audio- Based on Multimodal Representation Fusion Sen Chen Zhilei Liu Jiaxing Liu Longbiao Wang 36 6 0 27 Apr 2022
Leveraging Deepfakes to Close the Domain Gap between Real and Synthetic Images in Facial Capture Pipelines Winnie Lin Yilin Zhu Demi Guo Ronald Fedkiw 18 3 0 22 Apr 2022
Do You Really Mean That? Content Driven Audio-Visual Deepfake Dataset and Multimodal Method for Temporal Forgery Localization Zhixi Cai Kalin Stefanov Abhinav Dhall Munawar Hayat 20 3 0 13 Apr 2022
Dynamic Neural Textures: Generating Talking-Face Videos with Continuously Controllable Expressions Zipeng Ye Zhiyao Sun Yu-Hui Wen Y. Sun Tian Lv Ran Yi Yong-Jin Liu CVBM 19 7 0 13 Apr 2022
Audio-Visual Person-of-Interest DeepFake Detection D. Cozzolino Alessandro Pianese Matthias Nießner L. Verdoliva 36 60 0 06 Apr 2022
VocaLiST: An Audio-Visual Synchronisation Model for Lips and Voices V. S. Kadandale Juan F. Montesinos G. Haro 27 23 0 05 Apr 2022
Lip to Speech Synthesis with Visual Context Attentional GAN Minsu Kim Joanna Hong Y. Ro 25 51 0 04 Apr 2022
VoiceMe: Personalized voice generation in TTS Pol van Rijn Silvan Mertes Dominik Schiller Piotr Dura Hubert Siuzdak Peter M. C. Harrison Elisabeth André Nori Jacoby 27 9 0 29 Mar 2022
Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation Xian Liu Qianyi Wu Hang Zhou Yinghao Xu Rui Qian Xinyi Lin Xiaowei Zhou Wayne Wu Bo Dai Bolei Zhou SLR 37 99 0 24 Mar 2022
On the role of Lip Articulation in Visual Speech Perception Zakaria Aldeneh Masha Fedzechkina Skyler Seto Katherine Metcalf Miguel Sarabia N. Apostoloff B. Theobald 24 1 0 18 Mar 2022
DialogueNeRF: Towards Realistic Avatar Face-to-Face Conversation Video Generation Yichao Yan Zanwei Zhou Zi Wang Chen-Ning Yang Xiaokang Yang CVBM 21 18 0 15 Mar 2022
An Audio-Visual Attention Based Multimodal Network for Fake Talking Face Videos Detection Gang Wang Peng Zhang Lei Xie Wei Huang Yufei Zha Yanni Zhang CVBM 21 5 0 10 Mar 2022
StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN Fei Yin Yong Zhang Xiaodong Cun Ming Cao Yanbo Fan Xuanxia Wang Qingyan Bai Baoyuan Wu Jue Wang Yujiu Yang CVBM 43 171 0 08 Mar 2022
Attention-Based Lip Audio-Visual Synthesis for Talking Face Generation in the Wild Gang Wang Peng Zhang Lei Xie Wei Huang Yufei Zha CVBM 24 14 0 08 Mar 2022
Freeform Body Motion Generation from Speech Jing-Fen Xu Wei Zhang Yalong Bai Qi-Biao Sun Tao Mei SLR 39 18 0 04 Mar 2022
Voice-Face Homogeneity Tells Deepfake Harry Cheng Yangyang Guo Tianyi Wang Qi Li Xiaojun Chang Liqiang Nie CVBM 31 67 0 04 Mar 2022
Human Detection of Political Speech Deepfakes across Transcripts, Audio, and Video Matthew Groh Aruna Sankaranarayanan Nikhil Singh Dong Young Kim A. Lippman Rosalind W. Picard 11 17 0 25 Feb 2022
Deepfake Detection for Facial Images with Facemasks D. Ko Sangjun Lee Jin-Yeol Park Saebyeol Shin Do-Soon Hong Simon S. Woo CVBM 32 7 0 23 Feb 2022
Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation Xian Liu Yinghao Xu Qianyi Wu Hang Zhou Wayne Wu Bolei Zhou VGen DiffM 3DH 45 140 0 19 Jan 2022
Towards Realistic Visual Dubbing with Heterogeneous Sources Tianyi Xie Liucheng Liao Cheng Bi Benlai Tang Xiang Yin Jianfei Yang Mingjie Wang Jiali Yao Yang Zhang Zejun Ma VGen 30 37 0 17 Jan 2022
Audio-Driven Talking Face Video Generation with Dynamic Convolution Kernels Zipeng Ye Mengfei Xia Ran Yi Juyong Zhang Yu-Kun Lai Xuanteng Huang Guoxin Zhang Yong-jin Liu CVBM 22 39 0 16 Jan 2022
DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Rendering Shunyu Yao Ruizhe Zhong Yichao Yan Guangtao Zhai Xiaokang Yang CVBM 24 90 0 03 Jan 2022
Multimodal Image Synthesis and Editing: The Generative AI Era Fangneng Zhan Yingchen Yu Rongliang Wu Jiahui Zhang Shijian Lu Lingjie Liu Adam Kortylewski Christian Theobalt Eric Xing EGVM 29 48 0 27 Dec 2021
Responsive Listening Head Generation: A Benchmark Dataset and Baseline Mohan Zhou Yalong Bai Wei Zhang Ting Yao T. Zhao Tao Mei EGVM 30 45 0 27 Dec 2021
Watch Those Words: Video Falsification Detection Using Word-Conditioned Facial Motion S. Agarwal Liwen Hu Evonne Ng Trevor Darrell Hao Li Anna Rohrbach AAML 31 19 0 21 Dec 2021
FaceFormer: Speech-Driven 3D Facial Animation with Transformers Yingruo Fan Zhaojiang Lin Jun Saito Wenping Wang Taku Komura CVBM 52 195 0 10 Dec 2021
One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning Suzhe Wang Lincheng Li Yueqing Ding Xin Yu CVBM 69 117 0 06 Dec 2021
Joint Audio-Text Model for Expressive Speech-Driven 3D Facial Animation Yingruo Fan Zhaojiang Lin Jun Saito Wenping Wang Taku Komura 31 21 0 04 Dec 2021
More than Words: In-the-Wild Visually-Driven Prosody for Text-to-Speech Michael Hassid Michelle Tadmor Ramanovich Brendan Shillingford Miaosen Wang Ye Jia Tal Remez DiffM 19 16 0 19 Nov 2021
Impact of Benign Modifications on Discriminative Performance of Deepfake Detectors Yuhang Lu Evgeniy Upenik Touradj Ebrahimi AAML 36 0 0 14 Nov 2021
Personalized One-Shot Lipreading for an ALS Patient Bipasha Sen Aditya Agarwal Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar LM&MA 6 3 0 02 Nov 2021
Imitating Arbitrary Talking Style for Realistic Audio-DrivenTalking Face Synthesis Haozhe Wu Jia Jia Haoyu Wang Yishun Dou Chao Duan Qingshan Deng CVBM 11 73 0 30 Oct 2021
Talking Head Generation with Audio and Speech Related Facial Action Units Sen Chen Zhilei Liu Jiaxing Liu Zhengxiang Yan Longbiao Wang CVBM 21 14 0 19 Oct 2021
Intelligent Video Editing: Incorporating Modern Talking Face Generation Algorithms in a Video Editor Anchit Gupta Faizan Farooq Khan Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar CVBM 24 6 0 16 Oct 2021
Neural Dubber: Dubbing for Videos According to Scripts Chenxu Hu Qiao Tian Tingle Li Yuping Wang Yuxuan Wang Hang Zhao DiffM VGen 36 39 0 15 Oct 2021
VisualTTS: TTS with Accurate Lip-Speech Synchronization for Automatic Voice Over Junchen Lu Berrak Sisman Rui Liu Mingyang Zhang Haizhou Li DiffM 34 19 0 07 Oct 2021
Invertible Frowns: Video-to-Video Facial Emotion Translation Ian H. Magnusson Aruna Sankaranarayanan A. Lippman VGen 32 6 0 16 Sep 2021
Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors Hasam Khalid Minhan Kim Shahroz Tariq Simon S. Woo 23 82 0 07 Sep 2021
Deep Person Generation: A Survey from the Perspective of Face, Pose and Cloth Synthesis Tong Sha Wei Zhang T. Shen Zhoujun Li Tao Mei 35 38 0 05 Sep 2021
Speech Drives Templates: Co-Speech Gesture Synthesis with Learned Templates Shenhan Qian Zhi Tu Yihao Zhi Wen Liu Shenghua Gao SLR 16 71 0 18 Aug 2021
FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learning Chenxu Zhang Yifan Zhao Yifei Huang Ming Zeng Saifeng Ni M. Budagavi Xiaohu Guo CVBM 18 122 0 18 Aug 2021
FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset Hasam Khalid Shahroz Tariq Minha Kim Simon S. Woo 36 185 0 11 Aug 2021
AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Person Xinsheng Wang Qicong Xie Jihua Zhu Lei Xie O. Scharenborg 31 16 0 09 Aug 2021
Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion Suzhe Wang Lincheng Li Yu-qiong Ding Changjie Fan Xin Yu VGen 41 160 0 20 Jul 2021
Parallel and High-Fidelity Text-to-Lip Generation Jinglin Liu Zhiying Zhu Yi Ren Wencan Huang Baoxing Huai N. Yuan Zhou Zhao 32 10 0 14 Jul 2021
Extending Text-to-Speech Synthesis with Articulatory Movement Prediction using Ultrasound Tongue Imaging Tamás Gábor Csapó 11 2 0 12 Jul 2021
Speech2Video: Cross-Modal Distillation for Speech to Video Generation Shijing Si Jianzong Wang Xiaoyang Qu Ning Cheng Wenqi Wei Xinghua Zhu Jing Xiao VGen 24 15 0 10 Jul 2021
NWT: Towards natural audio-to-video generation with representation learning Rayhane Mama Marc S. Tyndel Hashiam Kadhim Cole Clifford Ragavan Thurairatnam VGen 29 12 0 08 Jun 2021
LipSync3D: Data-Efficient Learning of Personalized 3D Talking Faces from Video using Pose and Lighting Normalization A. Lahiri Vivek Kwatra C. Frueh J. P. Lewis C. Bregler 3DH 38 99 0 08 Jun 2021
Deepfake Detection by Human Crowds, Machines, and Machine-informed Crowds Matthew Groh Ziv Epstein C. Firestone Rosalind W. Picard 49 144 0 13 May 2021