OpFlowTalker: Realistic and Natural Talking Face Generation via Optical Flow Guidance

23 May 2024

Papers citing "OpFlowTalker: Realistic and Natural Talking Face Generation via Optical Flow Guidance"

46 / 46 papers shown

Title
FlowVQTalker: High-Quality Emotional Talking Face Generation through Normalizing Flow and Quantization Shuai Tan Bin Ji Ye Pan 61 15 0 11 Mar 2024
Style2Talker: High-Resolution Talking Head Generation with Emotion Style and Art Style Shuai Tan Bin Ji Ye Pan 58 20 0 11 Mar 2024
Say Anything with Any Style Shuai Tan Bin Ji Yu Ding Ye Pan VGen DiffM 52 10 0 11 Mar 2024
DreamTalk: When Expressive Talking Head Generation Meets Diffusion Probabilistic Models Yifeng Ma Shiwei Zhang Jiayu Wang Xiang Wang Yingya Zhang Zhidong Deng DiffM 83 23 0 15 Dec 2023
SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis Ziqiao Peng Wentao Hu Yue Shi Xiangyu Zhu Xiaomei Zhang Hao Zhao Jun He Hongyan Liu Zhaoxin Fan 68 42 0 29 Nov 2023
PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance Peiqing Yang Shangchen Zhou Qingyi Tao Chen Change Loy DiffM 34 31 0 19 Sep 2023
SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces Ziqiao Peng Yihao Luo Yue Shi Hao-Xuan Xu Xiangyu Zhu Jun He Hongyan Liu Zhaoxin Fan 70 42 0 19 Jun 2023
Identity-Preserving Talking Face Generation with Landmark and Appearance Priors Wei‐Tao Zhong Chaowei Fang Yinqi Cai Pengxu Wei Gangming Zhao Liang Lin Guanbin Li 46 75 0 15 May 2023
DAE-Talker: High Fidelity Speech-Driven Talking Face Generation with Diffusion Autoencoder Chenpeng Du Qi Chen Xie Chen K. Yu DiffM 77 51 0 30 Mar 2023
Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expert Jiadong Wang Xinyuan Qian Malu Zhang R. Tan Haizhou Li EGVM 48 97 0 29 Mar 2023
DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation Shuai Shen Wenliang Zhao Zibin Meng Wanhua Li Zhengbiao Zhu Jie Zhou Jiwen Lu DiffM VGen 63 104 0 10 Jan 2023
Memories are One-to-Many Mapping Alleviators in Talking Face Generation Anni Tang Tianyu He Xuejiao Tan Jun Ling Liang Song CVBM 50 23 0 09 Dec 2022
VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild K. Cheng Xiaodong Cun Yong Zhang Menghan Xia Fei Yin Mingrui Zhu Xuanxia Wang Jue Wang Nan Wang CVBM 45 96 0 27 Nov 2022
SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation Wenxuan Zhang Xiaodong Cun Xuan Wang Yong Zhang Xiaodong Shen Yu-Xiao Guo Ying Shan Fei Wang VGen 68 240 0 22 Nov 2022
Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis Shuai Shen Wanhua Li Zhengbiao Zhu Yueqi Duan Jie Zhou Jiwen Lu CVBM 71 106 0 24 Jul 2022
Visual Speech Recognition for Multiple Languages in the Wild Pingchuan Ma Stavros Petridis Maja Pantic VLM 144 147 0 26 Feb 2022
Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation Xian Liu Yinghao Xu Qianyi Wu Hang Zhou Wayne Wu Bolei Zhou VGen DiffM 3DH 69 142 0 19 Jan 2022
Robust Self-Supervised Audio-Visual Speech Recognition Bowen Shi Wei-Ning Hsu Abdel-rahman Mohamed 48 92 0 05 Jan 2022
Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction Bowen Shi Wei-Ning Hsu Kushal Lakhotia Abdel-rahman Mohamed SSL 82 310 0 05 Jan 2022
One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning Suzhe Wang Lincheng Li Yueqing Ding Xin Yu CVBM 82 119 0 06 Dec 2021
Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion Suzhe Wang Lincheng Li Yu-qiong Ding Changjie Fan Xin Yu VGen 80 163 0 20 Jul 2021
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation Hang Zhou Yasheng Sun Wayne Wu Chen Change Loy Xiaogang Wang Ziwei Liu CVBM 89 363 0 22 Apr 2021
Learning to Estimate Hidden Motions with Global Motion Aggregation Shihao Jiang Dylan Campbell Yao Lu Hongdong Li Leonid Sigal 3DPC 101 293 0 06 Apr 2021
Pre-training strategies and datasets for facial representation learning Adrian Bulat Shiyang Cheng Jing Yang A. Garbett Enrique Sanchez Georgios Tzimiropoulos CVBM SSL 47 36 0 30 Mar 2021
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild Prajwal K R Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar EGVM 96 772 0 23 Aug 2020
Talking-head Generation with Rhythmic Head Motion Lele Chen Guofeng Cui Celong Liu Zhong Li Ziyi Kou Yi Tian Xu Chenliang Xu 52 182 0 16 Jul 2020
MakeItTalk: Speaker-Aware Talking-Head Animation Yang Zhou Xintong Han Eli Shechtman J. Echevarria E. Kalogerakis Dingzeyu Li 61 417 0 27 Apr 2020
RAFT: Recurrent All-Pairs Field Transforms for Optical Flow Zachary Teed Jia Deng MDE 207 2,583 0 26 Mar 2020
Towards Automatic Face-to-Face Translation Prajwal K R Rudrabha Mukhopadhyay Jerin Philip Abhishek Jha Vinay P. Namboodiri C. V. Jawahar CVBM 86 173 0 01 Mar 2020
First Order Motion Model for Image Animation Aliaksandr Siarohin Stéphane Lathuilière Sergey Tulyakov Elisa Ricci N. Sebe VGen DiffM 77 918 0 29 Feb 2020
Neural Voice Puppetry: Audio-driven Facial Reenactment Justus Thies Mohamed A. Elgharib A. Tewari Christian Theobalt Matthias Nießner VGen 3DH 80 369 0 11 Dec 2019
Few-Shot Adversarial Learning of Realistic Neural Talking Head Models Egor Zakharov Aliaksandra Shysheya Egor Burkov Victor Lempitsky 3DH 144 629 0 20 May 2019
Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Loss Lele Chen R. Maddox Z. Duan Chenliang Xu CVBM 66 395 0 09 May 2019
Deep Audio-Visual Speech Recognition Triantafyllos Afouras Joon Son Chung A. Senior Oriol Vinyals Andrew Zisserman 62 696 0 06 Sep 2018
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation Hang Zhou Yu Liu Ziwei Liu Ping Luo Xiaogang Wang CVBM 78 438 0 20 Jul 2018
Deep Video Portraits Hyeongwoo Kim Pablo Garrido A. Tewari Weipeng Xu Justus Thies Matthias Nießner P. Pérez Christian Richardt Michael Zollhöfer Christian Theobalt 3DH VGen 82 695 0 29 May 2018
Talking Face Generation by Conditional Recurrent Adversarial Network Yang Song Jingwen Zhu Dawei Li Xiaolong Wang Hairong Qi CVBM 120 194 0 13 Apr 2018
Lip Movements Generation at a Glance Lele Chen Zhiheng Li R. Maddox Z. Duan Chenliang Xu 111 263 0 28 Mar 2018
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric Richard Y. Zhang Phillip Isola Alexei A. Efros Eli Shechtman Oliver Wang EGVM 317 11,610 0 11 Jan 2018
StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks Han Zhang Tao Xu Hongsheng Li Shaoting Zhang Xiaogang Wang Xiaolei Huang Dimitris N. Metaxas GAN 77 1,055 0 19 Oct 2017
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume Deqing Sun Xiaodong Yang Ming-Yuan Liu Jan Kautz 3DPC 244 2,440 0 07 Sep 2017
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks Eddy Ilg N. Mayer Tonmoy Saikia Margret Keuper Alexey Dosovitskiy Thomas Brox 3DPC 212 3,072 0 06 Dec 2016
Lip Reading Sentences in the Wild Joon Son Chung A. Senior Oriol Vinyals Andrew Zisserman 245 788 0 16 Nov 2016
Deep Residual Learning for Image Recognition Kaiming He Xinming Zhang Shaoqing Ren Jian Sun MedIm 1.6K 192,638 0 10 Dec 2015
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin Dario Amodei Rishita Anubhai Eric Battenberg Carl Case Jared Casper ... Chong-Jun Wang Bo Xiao Dani Yogatama J. Zhan Zhenyao Zhu 116 2,965 0 08 Dec 2015
FlowNet: Learning Optical Flow with Convolutional Networks Philipp Fischer Alexey Dosovitskiy Eddy Ilg Philip Häusser C. Hazirbas Vladimir Golkov Patrick van der Smagt Daniel Cremers Thomas Brox 3DPC 278 4,159 0 26 Apr 2015