StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based Generator

9 May 2023

Errui Ding

Ziwei Liu

Jingdong Wang

ArXiv (abs)PDF HTML

Papers citing "StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based Generator"

50 / 50 papers shown

Title
MagicPortrait: Temporally Consistent Face Reenactment with 3D Geometric Guidance Mengting Wei Yante Li Tuomas Varanka Yan Jiang Guoying Zhao DiffM VGen 144 1 0 30 Apr 2025
HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priors Xiaozheng Zheng Chao Wen Zhaohu Li Weiyi Zhang Zhuo Su ... Zheng Lv Xiaoyuan Zhang Yongjie Zhang Guidong Wang Lan Xu 3DGS 131 12 0 12 Aug 2024
DiffusionAct: Controllable Diffusion Autoencoder for One-shot Face Reenactment Stella Bounareli Christos Tzelepis Vasileios Argyriou Ioannis Patras Georgios Tzimiropoulos DiffM 107 7 0 25 Mar 2024
GMTalker: Gaussian Mixture-based Audio-Driven Emotional Talking Video Portraits Yibo Xia Lizhen Wang Xiang Deng Xiaoyan Luo Yunhong Wang Yebin Liu VGen 114 1 0 12 Dec 2023
GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis Zhenhui Ye Ziyue Jiang Yi Ren Jinglin Liu Jinzheng He Zhou Zhao CVBM 82 128 0 31 Jan 2023
StyleTalk: One-shot Talking Head Generation with Controllable Speaking Styles Yifeng Ma Suzhe Wang Zhipeng Hu Changjie Fan Tangjie Lv Yu-qiong Ding Zhidong Deng Xin Yu 120 89 0 03 Jan 2023
Masked Lip-Sync Prediction by Audio-Visual Contextual Exploitation in Transformers Yasheng Sun Hang Zhou Kaisiyuan Wang Qianyi Wu Zhibin Hong Jingtuo Liu Errui Ding Jingdong Wang Ziwei Liu Koike Hideki 60 34 0 09 Dec 2022
Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decomposition Jiaxiang Tang Kaisiyuan Wang Hang Zhou Xiaokang Chen Dongliang He Tianshu Hu Jingtuo Liu Gang Zeng Jingdong Wang 3DH 72 80 0 22 Nov 2022
SPACE: Speech-driven Portrait Animation with Controllable Expression Francesco Ferroni Arun Mallya Ting-Chun Wang Rafael Valle Xuan Li VGen 70 47 0 17 Nov 2022
SyncTalkFace: Talking Face Generation with Precise Lip-Syncing via Audio-Lip Memory Se Jin Park Minsu Kim Joanna Hong J. Choi Y. Ro CVBM 118 90 0 02 Nov 2022
StyleSwap: Style-Based Generator Empowers Robust Face Swapping Zhi-liang Xu Hang Zhou Zhibin Hong Ziwei Liu Jiaming Liu Zhizhi Guo Junyu Han Jingtuo Liu Errui Ding Jingdong Wang CVBM 96 46 0 27 Sep 2022
Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis Shuai Shen Wanhua Li Zhengbiao Zhu Yueqi Duan Jie Zhou Jiwen Lu CVBM 82 107 0 24 Jul 2022
EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model Xinya Ji Hang Zhou Kaisiyuan Wang Qianyi Wu Wayne Wu Feng Xu Xun Cao CVBM 132 165 0 30 May 2022
Region-Aware Face Swapping Chao Xu Jiangning Zhang Miao Hua Qian He Zili Yi Yong Liu CVBM 75 50 0 09 Mar 2022
StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN Fei Yin Yong Zhang Xiaodong Cun Ming Cao Yanbo Fan Xuanxia Wang Qingyan Bai Baoyuan Wu Jue Wang Yujiu Yang CVBM 113 174 0 08 Mar 2022
FaceFormer: Speech-Driven 3D Facial Animation with Transformers Yingruo Fan Zhaojiang Lin Jun Saito Wenping Wang Taku Komura CVBM 90 205 0 10 Dec 2021
One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning Suzhe Wang Lincheng Li Yueqing Ding Xin Yu CVBM 114 119 0 06 Dec 2021
HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing Yuval Alaluf Omer Tov Ron Mokady Rinon Gal Amit H. Bermano 133 276 0 30 Nov 2021
FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learning Chenxu Zhang Yifan Zhao Yifei Huang Ming Zeng Saifeng Ni M. Budagavi Xiaohu Guo CVBM 79 126 0 18 Aug 2021
Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion Suzhe Wang Lincheng Li Yu-qiong Ding Changjie Fan Xin Yu VGen 101 165 0 20 Jul 2021
Alias-Free Generative Adversarial Networks Tero Karras M. Aittala S. Laine Erik Härkönen Janne Hellsten J. Lehtinen Timo Aila GAN 185 1,603 0 23 Jun 2021
Pivotal Tuning for Latent-based Editing of Real Images Daniel Roich Ron Mokady Amit H. Bermano Daniel Cohen-Or DiffM 130 533 0 10 Jun 2021
LipSync3D: Data-Efficient Learning of Personalized 3D Talking Faces from Video using Pose and Lighting Normalization A. Lahiri Vivek Kwatra C. Frueh J. P. Lewis C. Bregler 3DH 81 101 0 08 Jun 2021
GAN Prior Embedded Network for Blind Face Restoration in the Wild Tao Yang Peiran Ren Xuansong Xie Lei Zhang GAN CVBM 85 279 0 13 May 2021
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation Hang Zhou Yasheng Sun Wayne Wu Chen Change Loy Xiaogang Wang Ziwei Liu CVBM 116 372 0 22 Apr 2021
MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement Alexander Richard Michael Zollhoefer Yandong Wen Fernando de la Torre Yaser Sheikh CVBM 68 200 0 16 Apr 2021
ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement Yuval Alaluf Or Patashnik Daniel Cohen-Or 110 347 0 06 Apr 2021
Designing an Encoder for StyleGAN Image Manipulation Omer Tov Yuval Alaluf Yotam Nitzan Or Patashnik Daniel Cohen-Or 295 789 0 04 Feb 2021
Towards Real-World Blind Face Restoration with Generative Facial Prior Xintao Wang Yu Li Honglun Zhang Ying Shan CVBM 94 455 0 11 Jan 2021
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild Prajwal K R Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar EGVM 103 793 0 23 Aug 2020
Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation Elad Richardson Yuval Alaluf Or Patashnik Yotam Nitzan Yaniv Azar Stav Shapiro Daniel Cohen-Or 152 1,109 0 03 Aug 2020
Talking-head Generation with Rhythmic Head Motion Lele Chen Guofeng Cui Celong Liu Zhong Li Ziyi Kou Yi Tian Xu Chenliang Xu 65 183 0 16 Jul 2020
Closed-Form Factorization of Latent Semantics in GANs Yujun Shen Bolei Zhou GAN 116 591 0 13 Jul 2020
InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs Yujun Shen Ceyuan Yang Xiaoou Tang Bolei Zhou GAN CVBM 79 600 0 18 May 2020
MakeItTalk: Speaker-Aware Talking-Head Animation Yang Zhou Xintong Han Eli Shechtman J. Echevarria E. Kalogerakis Dingzeyu Li 80 424 0 27 Apr 2020
Neural Head Reenactment with Latent Pose Descriptors Egor Burkov I. Pasechnik Artur Grigorev Victor Lempitsky 3DH 114 131 0 24 Apr 2020
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis B. Mildenhall Pratul P. Srinivasan Matthew Tancik Jonathan T. Barron R. Ramamoorthi Ren Ng 133 2,592 0 19 Mar 2020
Audio-driven Talking Face Video Generation with Learning-based Personalized Head Pose Ran Yi Zipeng Ye Juyong Zhang Hujun Bao Yong Liu CVBM 95 123 0 24 Feb 2020
Everybody's Talkin': Let Me Talk as You Want Linsen Song Wayne Wu Chao Qian Ran He Chen Change Loy DiffM VGen 79 147 0 15 Jan 2020
Analyzing and Improving the Image Quality of StyleGAN Tero Karras S. Laine M. Aittala Janne Hellsten J. Lehtinen Timo Aila GAN 334 5,832 0 03 Dec 2019
Image2StyleGAN++: How to Edit the Embedded Images? Rameen Abdal Yipeng Qin Peter Wonka 99 556 0 26 Nov 2019
Few-Shot Adversarial Learning of Realistic Neural Talking Head Models Egor Zakharov Aliaksandra Shysheya Egor Burkov Victor Lempitsky 3DH 149 631 0 20 May 2019
Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space? Rameen Abdal Yipeng Qin Peter Wonka GAN 114 1,112 0 05 Apr 2019
A Style-Based Generator Architecture for Generative Adversarial Networks Tero Karras S. Laine Timo Aila 628 10,595 0 12 Dec 2018
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation Hang Zhou Yu Liu Ziwei Liu Ping Luo Xiaogang Wang CVBM 92 443 0 20 Jul 2018
VoxCeleb2: Deep Speaker Recognition Joon Son Chung Arsha Nagrani Andrew Zisserman 358 2,287 0 14 Jun 2018
VisemeNet: Audio-Driven Animator-Centric Speech Animation Yang Zhou Shan Xu Chris Landreth E. Kalogerakis Subhransu Maji Karan Singh 44 146 0 24 May 2018
Talking Face Generation by Conditional Recurrent Adversarial Network Yang Song Jingwen Zhu Dawei Li Xiaolong Wang Hairong Qi CVBM 161 196 0 13 Apr 2018
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs Ting-Chun Wang Ming-Yuan Liu Jun-Yan Zhu Andrew Tao Jan Kautz Bryan Catanzaro SSeg 187 3,955 0 30 Nov 2017
VoxCeleb: a large-scale speaker identification dataset Arsha Nagrani Joon Son Chung Andrew Zisserman 129 2,287 0 26 Jun 2017