A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild

23 August 2020

Prajwal K R

Rudrabha Mukhopadhyay

Papers citing "A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild"

50 / 410 papers shown

Title
Unsupervised Multimodal Deepfake Detection Using Intra- and Cross-Modal Inconsistencies Mulin Tian Mahyar Khayatkhoei Joe Mathai Wael AbdAlmageed 35 6 0 28 Nov 2023
AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset Zhixi Cai Shreya Ghosh Aman Pankaj Adatia Munawar Hayat Abhinav Dhall Kalin Stefanov 21 27 0 26 Nov 2023
GAIA: Zero-shot Talking Avatar Generation Tianyu He Junliang Guo Runyi Yu Yuchi Wang Jialiang Zhu ... Chunyu Wang Han Hu HsiangTao Wu Sheng Zhao Jiang Bian 31 25 0 26 Nov 2023
Multimodal Large Language Models: A Survey Jiayang Wu Wensheng Gan Zefeng Chen Shicheng Wan Philip S. Yu 36 169 0 22 Nov 2023
CP-EB: Talking Face Generation with Controllable Pose and Eye Blinking Embedding Jianzong Wang Yimin Deng Ziqi Liang Xulong Zhang Ning Cheng Jing Xiao CVBM 21 2 0 15 Nov 2023
AV-Lip-Sync+: Leveraging AV-HuBERT to Exploit Multimodal Inconsistency for Video Deepfake Detection Sahibzada Adil Shahzad Ammarah Hashmi Yan-Tsung Peng Yu Tsao Hsin-Min Wang 32 5 0 05 Nov 2023
DiffDub: Person-generic Visual Dubbing Using Inpainting Renderer with Diffusion Auto-encoder Tao Liu Chenpeng Du Shuai Fan Feilong Chen Kai Yu DiffM VGen 17 6 0 03 Nov 2023
Detecting Deepfakes Without Seeing Any Tal Reiss Bar Cavia Yedid Hoshen AAML 28 17 0 02 Nov 2023
Deepfake detection by exploiting surface anomalies: the SurFake approach Andrea Ciamarra R. Caldelli Federico Becattini Lorenzo Seidenari A. Bimbo 36 14 0 31 Oct 2023
Breathing Life into Faces: Speech-driven 3D Facial Animation with Natural Head Pose and Detailed Shape Wei Zhao Yijun Wang Tianyu He Li-Ping Yin Jianxin Lin Xin Jin 3DH 29 2 0 31 Oct 2023
AVTENet: Audio-Visual Transformer-based Ensemble Network Exploiting Multiple Experts for Video Deepfake Detection Ammarah Hashmi Sahibzada Adil Shahzad Chia-Wen Lin Yu Tsao Hsin-Min Wang ViT 53 6 0 19 Oct 2023
CorrTalk: Correlation Between Hierarchical Speech and Facial Activity Variances for 3D Animation Zhaojie Chu K. Guo Xiaofen Xing Yilin Lan Bolun Cai Xiangmin Xu 43 5 0 17 Oct 2023
HyperLips: Hyper Control Lips with High Resolution Decoder for Talking Face Generation Yaosen Chen Yu Yao Zhiqiang Li Wei Wang Yanru Zhang Han Yang Xuming Wen 32 8 0 09 Oct 2023
OSM-Net: One-to-Many One-shot Talking Head Generation with Spontaneous Head Motions Jin Liu Xi Wang Xiaomeng Fu Yesheng Chai Cai Yu Jiao Dai Jizhong Han 21 3 0 28 Sep 2023
TRAVID: An End-to-End Video Translation Framework Prottay Kumar Adhikary Bandaru Sugandhi Subhojit Ghimire Santanu Pal Partha Pakray 19 2 0 20 Sep 2023
Locate and Verify: A Two-Stream Network for Improved Deepfake Detection Chao Shuai Jieming Zhong Shuang Wu Feng Lin Zhibo Wang Zhongjie Ba Zhenguang Liu Lorenzo Cavallaro Kui Ren 48 28 0 20 Sep 2023
DT-NeRF: Decomposed Triplane-Hash Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis Yaoyu Su Shaohui Wang Haoqian Wang CVBM 33 2 0 14 Sep 2023
DiffTalker: Co-driven audio-image diffusion for talking faces via intermediate landmarks Zipeng Qi Xulong Zhang Ning Cheng Jing Xiao Jianzong Wang 24 7 0 14 Sep 2023
HDTR-Net: A Real-Time High-Definition Teeth Restoration Network for Arbitrary Talking Face Generation Methods Yongyuan Li Xiuyuan Qin Chao Liang Mingqiang Wei 27 3 0 14 Sep 2023
Efficient Emotional Adaptation for Audio-Driven Talking-Head Generation Yuan Gan Zongxin Yang Xihang Yue Lingyun Sun Yezhou Yang 25 57 0 10 Sep 2023
Speech2Lip: High-fidelity Speech to Lip Generation by Learning from a Short Video Xiuzhe Wu Pengfei Hu Yang Wu Xiaoyang Lyu Yan-Pei Cao Ying Shan Wenming Yang Zhongqian Sun Xiaojuan Qi 23 14 0 09 Sep 2023
The Power of Sound (TPoS): Audio Reactive Video Generation with Stable Diffusion Yujin Jeong Won-Wha Ryoo Seunghyun Lee Dabin Seo Wonmin Byeon Sangpil Kim Jinkyu Kim DiffM 24 29 0 08 Sep 2023
ReliTalk: Relightable Talking Portrait Generation from a Single Video Haonan Qiu Zhaoxi Chen Yuming Jiang Hang Zhou Xiangyu Fan Lei Yang Wayne Wu Ziwei Liu DiffM VGen 34 10 0 05 Sep 2023
RADIO: Reference-Agnostic Dubbing Video Synthesis Dongyeun Lee Chaewon Kim Sangjoon Yu Jaejun Yoo Gyeong-Moon Park VGen DiffM 42 1 0 05 Sep 2023
Audio-Driven Dubbing for User Generated Contents via Style-Aware Semi-Parametric Synthesis Linsen Song Wayne Wu Chaoyou Fu Chen Change Loy Ran He 31 10 0 31 Aug 2023
MFR-Net: Multi-faceted Responsive Listening Head Generation via Denoising Diffusion Model Jin Liu Xi Wang Xiaomeng Fu Yesheng Chai Cai Yu Jiao Dai Jizhong Han DiffM 29 10 0 31 Aug 2023
From Pixels to Portraits: A Comprehensive Survey of Talking Head Generation Techniques and Applications Shreyank N. Gowda Dheeraj Pandey Shashank Narayana Gowda 52 3 0 30 Aug 2023
FaceChain: A Playground for Human-centric Artificial Intelligence Generated Content Yang Liu Cheng Yu Lei Shang Yongyi He Ziheng Wu ... Jiaqi Xu Qiang Wang Yingda Chen Xuansong Xie Baigui Sun 41 5 0 28 Aug 2023
AI-Generated Content (AIGC) for Various Data Modalities: A Survey Lin Geng Foo Hossein Rahmani Xiaozhong Liu 78 31 0 27 Aug 2023
AdVerb: Visually Guided Audio Dereverberation Sanjoy Chowdhury Sreyan Ghosh Subhrajyoti Dasgupta Anton Ratnarajah Utkarsh Tyagi Tianyi Zhou 30 11 0 23 Aug 2023
DF-3DFace: One-to-Many Speech Synchronized 3D Face Animation with Diffusion Se Jin Park Joanna Hong Minsu Kim Y. Ro 37 4 0 23 Aug 2023
Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization Soumik Mukhopadhyay Saksham Suri R. Gadde Abhinav Shrivastava DiffM 46 20 0 18 Aug 2023
A Survey on Deep Multi-modal Learning for Body Language Recognition and Generation Li Liu Lufei Gao Wen-Ling Lei Fengji Ma Xiaotian Lin Jin-Tao Wang CVBM 27 5 0 17 Aug 2023
Controlling Character Motions without Observable Driving Source Weiyuan Li Bin Dai Ziyi Zhou Qi Yao Baoyuan Wang VGen 8 1 0 11 Aug 2023
Speech-Driven 3D Face Animation with Composite and Regional Facial Movements Haozhe Wu Songtao Zhou Jia Jia Junliang Xing Qi Wen Xiang Wen CVBM 32 15 0 10 Aug 2023
VAST: Vivify Your Talking Avatar via Zero-Shot Expressive Facial Style Transfer Liyang Chen Zhiyong Wu Runnan Li Weihong Bao Jun Ling Xuejiao Tan Sheng Zhao 29 5 0 09 Aug 2023
Deepfake Detection: A Comparative Analysis Sohail Ahmed Khan Duc-Tien Dang-Nguyen 36 2 0 07 Aug 2023
A Unified Framework for Modality-Agnostic Deepfakes Detection Cai Yu Peng-Wen Chen Jiahe Tian Jin Liu Jiao Dai Xi Wang Yesheng Chai Shan Jia Siwei Lyu Jizhong Han 32 0 0 26 Jul 2023
Learning and Evaluating Human Preferences for Conversational Head Generation Mohan Zhou Yalong Bai Wei Zhang Ting Yao Tiejun Zhao Tao Mei 32 2 0 20 Jul 2023
MODA: Mapping-Once Audio-driven Portrait Animation with Dual Attentions Yunfei Liu Lijian Lin Fei Yu Changyin Zhou Yu Li DiffM VGen 42 23 0 19 Jul 2023
Hierarchical Semantic Perceptual Listener Head Video Generation: A High-performance Pipeline Zhigang Chang Weitai Hu Q. Yang Shibao Zheng VGen 16 5 0 19 Jul 2023
FACTS: Facial Animation Creation using the Transfer of Styles Jack D. Saunders Steven Caulkin Vinay P. Namboodiri 3DH CVBM 47 0 0 18 Jul 2023
Audio-driven Talking Face Generation with Stabilized Synchronization Loss Dogucan Yaman Fevziye Irem Eyiokur Leonard Barmann H. K. Ekenel Alexander Waibel CVBM 40 3 0 18 Jul 2023
Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis Jiahe Li Jiawei Zhang Xiao Bai Jun Zhou L. Gu 3DH 26 62 0 18 Jul 2023
OPHAvatars: One-shot Photo-realistic Head Avatars Shaoxu Li 37 1 0 18 Jul 2023
FTFDNet: Learning to Detect Talking Face Video Manipulation with Tri-Modality Interaction Gang Wang Peng Zhang Jun Xiong Fei Yang Wei Huang Yufei Zha CVBM 25 1 0 08 Jul 2023
Interactive Conversational Head Generation Mohan Zhou Yalong Bai Wei Zhang Tingjun Yao Tiejun Zhao 27 3 0 05 Jul 2023
A Comprehensive Multi-scale Approach for Speech and Dynamics Synchrony in Talking Head Generation Louis Airale Dominique Vaufreydaz Xavier Alameda-Pineda 23 1 0 04 Jul 2023
High-Quality Automatic Voice Over with Accurate Alignment: Supervision through Self-Supervised Discrete Speech Units Junchen Lu Berrak Sisman Mingyang Zhang Haizhou Li 24 4 0 29 Jun 2023
Text-driven Talking Face Synthesis by Reprogramming Audio-driven Models J. Choi Minsu Kim Se Jin Park Y. Ro CVBM 16 3 0 28 Jun 2023