Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization

Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization

18 August 2023

Soumik Mukhopadhyay

Abhinav Shrivastava

Papers citing "Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization"

18 / 18 papers shown

Title
GenSync: A Generalized Talking Head Framework for Audio-driven Multi-Subject Lip-Sync using 3D Gaussian Splatting Anushka Agarwal Muhammad Yusuf Hassan Talha Chafekar 3DGS 26 0 0 03 May 2025
KeySync: A Robust Approach for Leakage-free Lip Synchronization in High Resolution Antoni Bigata Rodrigo Mira Stella Bounareli Michał Stypułkowski Konstantinos Vougioukas Stavros Petridis Maja Pantic 52 0 0 01 May 2025
FluentLip: A Phonemes-Based Two-stage Approach for Audio-Driven Lip Synthesis with Optical Flow Consistency Shiyan Liu Rui Qu Yan Jin 31 0 0 06 Apr 2025
Detecting Lip-Syncing Deepfakes: Vision Temporal Transformer for Analyzing Mouth Inconsistencies Soumyya Kanti Datta Shan Jia Siwei Lyu 44 0 0 02 Apr 2025
RASA: Replace Anyone, Say Anything -- A Training-Free Framework for Audio-Driven and Universal Portrait Video Editing Tianrui Pan Lin Liu Jie Liu X. Zhang J. Tang Gangshan Wu Q. Tian DiffM VGen 51 0 0 14 Mar 2025
Removing Averaging: Personalized Lip-Sync Driven Characters Based on Identity Adapter Yanyu Zhu Licheng Bai Jintao Xu Jiwei Tang Hai-tao Zheng 38 0 0 09 Mar 2025
FREAK: Frequency-modulated High-fidelity and Real-time Audio-driven Talking Portrait Synthesis Ziqi Ni Ao Fu Yi Zhou 61 0 0 06 Mar 2025
SayAnything: Audio-Driven Lip Synchronization with Conditional Video Diffusion Junxian Ma Shiwen Wang Jian Yang Junyi Hu Jian Liang Guosheng Lin Jingbo Chen Kai Li Yu Meng DiffM VGen 61 3 0 17 Feb 2025
GLCF: A Global-Local Multimodal Coherence Analysis Framework for Talking Face Generation Detection Xiaocan Chen Qilin Yin Jiarui Liu Wei Lu Xiangyang Luo Jiantao Zhou CVBM 84 0 0 18 Dec 2024
High-fidelity and Lip-synced Talking Face Synthesis via Landmark-based Diffusion Model Weizhi Zhong Junfan Lin Peixin Chen Liang Lin Guanbin Li 34 1 0 10 Aug 2024
SwapTalk: Audio-Driven Talking Face Generation with One-Shot Customization in Latent Space Zeren Zhang Haibo Qin Jiayu Huang Yixin Li Hui Lin Yitao Duan Jinwen Ma 35 0 0 09 May 2024
Superior and Pragmatic Talking Face Generation with Teacher-Student Framework Chao Liang Jianwen Jiang Tianyun Zhong Gaojie Lin Zhengkun Rong Jiaqi Yang Yongming Zhu 37 1 0 26 Mar 2024
EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions Linrui Tian Qi Wang Bang Zhang Liefeng Bo DiffM 61 101 0 27 Feb 2024
DreamTalk: When Expressive Talking Head Generation Meets Diffusion Probabilistic Models Yifeng Ma Shiwei Zhang Jiayu Wang Xiang Wang Yingya Zhang Zhidong Deng DiffM 39 23 0 15 Dec 2023
Pretraining is All You Need for Image-to-Image Translation Tengfei Wang Ting Zhang Bo Zhang Hao Ouyang Dong Chen Qifeng Chen Fang Wen DiffM 189 178 0 25 May 2022
Palette: Image-to-Image Diffusion Models Chitwan Saharia William Chan Huiwen Chang Chris A. Lee Jonathan Ho Tim Salimans David J. Fleet Mohammad Norouzi DiffM VLM 342 1,588 0 10 Nov 2021
VoxCeleb2: Deep Speaker Recognition Joon Son Chung Arsha Nagrani Andrew Zisserman 219 2,233 0 14 Jun 2018
Image-to-Image Translation with Conditional Adversarial Networks Phillip Isola Jun-Yan Zhu Tinghui Zhou Alexei A. Efros SSeg 212 19,447 0 21 Nov 2016