Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decomposition

22 November 2022

Jingdong Wang

Papers citing "Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decomposition"

50 / 53 papers shown

Title
SE4Lip: Speech-Lip Encoder for Talking Head Synthesis to Solve Phoneme-Viseme Alignment Ambiguity Yihuan Huang Jiajun Liu Yanzhen Ren Wuyang Liu Juhua Tang 24 0 0 08 Apr 2025
DiTaiListener: Controllable High Fidelity Listener Video Generation with Diffusion Maksim Siniukov Di Chang Minh Tran Hongkun Gong Ashutosh Chaubey Mohammad Soleymani DiffM VGen 23 0 0 05 Apr 2025
Dual Audio-Centric Modality Coupling for Talking Head Generation Ao Fu Ziqi Ni Yi Zhou 37 1 0 26 Mar 2025
EmoHead: Emotional Talking Head via Manipulating Semantic Expression Parameters Xuli Shen Hua Cai Dingding Yu Weilin Shen Qing-Song Xu Xiangyang Xue 34 0 0 25 Mar 2025
FREAK: Frequency-modulated High-fidelity and Real-time Audio-driven Talking Portrait Synthesis Ziqi Ni Ao Fu Yi Zhou 61 0 0 06 Mar 2025
InsTaG: Learning Personalized 3D Talking Head from Few-Second Video Jiahe Li Jiawei Zhang Xiao Bai Jin Zheng J. Zhou L. Gu 62 0 0 27 Feb 2025
LAM: Large Avatar Model for One-shot Animatable Gaussian Head Yisheng He Xiaodong Gu Xiaodan Ye Chao Xu Zhengyi Zhao Yuan Dong Weihao Yuan Zilong Dong Liefeng Bo 3DGS 85 0 0 25 Feb 2025
DEGSTalk: Decomposed Per-Embedding Gaussian Fields for Hair-Preserving Talking Face Synthesis Kaijun Deng Dezhi Zheng Jindong Xie Jinbao Wang Weicheng Xie L. Shen Siyang Song 3DGS 32 0 0 31 Dec 2024
FADA: Fast Diffusion Avatar Synthesis with Mixed-Supervised Multi-CFG Distillation Tianyun Zhong Chao Liang Jianwen Jiang Gaojie Lin Jiaqi Yang Zhou Zhao DiffM 99 1 0 22 Dec 2024
Generalizable and Animatable Gaussian Head Avatar Xuangeng Chu Tatsuya Harada 3DGS 36 19 0 10 Oct 2024
MimicTalk: Mimicking a personalized and expressive 3D talking face in minutes Zhenhui Ye Tianyun Zhong Yi Ren Ziyue Jiang Jiawei Huang ... Chen Zhang Zehan Wang Xize Chen Xiang Yin Zhou Zhao VGen 36 3 0 09 Oct 2024
LaDTalk: Latent Denoising for Synthesizing Talking Head Videos with High Frequency Details Jian Yang Xukun Wang Wentao Wang Guoming Li Qihang Fang Ruihong Yuan Tianyang Wang Jason Zhaoxin Fan Yeying Jin Zhaoxin Fan VGen 47 1 0 01 Oct 2024
PersonaTalk: Bring Attention to Your Persona in Visual Dubbing Longhao Zhang Shuang Liang Zhipeng Ge Tianshu Hu DiffM VGen 29 5 0 09 Sep 2024
S^3D-NeRF: Single-Shot Speech-Driven Neural Radiance Field for High Fidelity Talking Head Synthesis Dongze Li Kang Zhao Wei Wang Yifeng Ma Bo Peng Yingya Zhang Jing Dong 3DH CVBM 35 2 0 18 Aug 2024
Synchronous Multi-modal Semantic Communication System with Packet-level Coding Yun Tian Jingkai Ying Zhijin Qin Ye Jin Xiaoming Tao 34 3 0 08 Aug 2024
ReSyncer: Rewiring Style-based Generator for Unified Audio-Visually Synced Facial Performer Jiazhi Guan Zhiliang Xu Hang Zhou Kaisiyuan Wang Shengyi He ... Errui Ding Jingtuo Liu Jingdong Wang Youjian Zhao Ziwei Liu VGen 51 2 0 06 Aug 2024
NLDF: Neural Light Dynamic Fields for Efficient 3D Talking Head Generation Niu Guanchen 3DH 47 0 0 17 Jun 2024
Controllable Talking Face Generation by Implicit Facial Keypoints Editing Dong Zhao Jiaying Shi Wenjun Li Shudong Wang Shenghui Xu Zhaoming Pan CVBM 46 0 0 05 Jun 2024
NeRFFaceSpeech: One-shot Audio-driven 3D Talking Head Synthesis via Generative Prior Gihoon Kim Kwanggyoon Seo Sihun Cha Junyong Noh DiffM 3DH 39 1 0 09 May 2024
Audio-Visual Speech Representation Expert for Enhanced Talking Face Video Generation and Evaluation Dogucan Yaman Fevziye Irem Eyiokur Leonard Barmann Seymanur Akti H. K. Ekenel Alexander H. Waibel EGVM 30 9 0 07 May 2024
TalkingGaussian: Structure-Persistent 3D Talking Head Synthesis via Gaussian Splatting Jiahe Li Jiawei Zhang Xiao Bai Jin Zheng Xin Ning Jun Zhou Lin Gu 3DGS 48 15 0 23 Apr 2024
Talk3D: High-Fidelity Talking Portrait Synthesis via Personalized 3D Generative Prior Jaehoon Ko Kyusun Cho Joungbin Lee Heeji Yoon Sangmin Lee Sangjun Ahn Seungryong Kim 34 2 0 29 Mar 2024
Learning Dynamic Tetrahedra for High-Quality Talking Head Synthesis Zicheng Zhang Ruobing Zheng Ziwen Liu Congying Han Tianqi Li Meng Wang Tiande Guo Jingdong Chen Bonan Li Ming Yang 3DH 32 5 0 27 Feb 2024
GPAvatar: Generalizable and Precise Head Avatar from Image(s) Xuangeng Chu Yu Li Ailing Zeng Tianyu Yang Lijian Lin Yunfei Liu Tatsuya Harada 3DGS 33 23 0 18 Jan 2024
$Tri$^{2}$-plane: Thinking Head Avatar via Feature Pyramid$ Tri $^{2}$ -plane: Thinking Head Avatar via Feature Pyramid Luchuan Song Pinxin Liu Lele Chen Guojun Yin Chenliang Xu 3DH 26 6 0 17 Jan 2024
Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis Zhenhui Ye Tianyun Zhong Yi Ren Jiaqi Yang Weichuang Li ... Jinglin Liu Chen Zhang Xiang Yin Zejun Ma Zhou Zhao 29 45 0 16 Jan 2024
Dubbing for Everyone: Data-Efficient Visual Dubbing using Neural Rendering Priors Jack D. Saunders Vinay P. Namboodiri VGen DiffM 31 1 0 11 Jan 2024
DreamTalk: When Expressive Talking Head Generation Meets Diffusion Probabilistic Models Yifeng Ma Shiwei Zhang Jiayu Wang Xiang Wang Yingya Zhang Zhidong Deng DiffM 44 2 0 15 Dec 2023
R2-Talker: Realistic Real-Time Talking Head Synthesis with Hash Grid Landmarks Encoding and Progressive Multilayer Conditioning Zhiling Ye LiangGuo Zhang Dingheng Zeng Quan Lu Ning Jiang 26 0 0 09 Dec 2023
GaussianHead: High-fidelity Head Avatars with Learnable Gaussian Derivation Jie Wang Jiucheng Xie Xianyan Li Feng Xu Chi-Man Pun Hao Gao 3DGS 41 23 0 04 Dec 2023
SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis Ziqiao Peng Wentao Hu Yue Shi Xiangyu Zhu Xiaomei Zhang Hao Zhao Jun He Hongyan Liu Zhaoxin Fan 38 39 0 29 Nov 2023
HyperLips: Hyper Control Lips with High Resolution Decoder for Talking Face Generation Yaosen Chen Yu Yao Zhiqiang Li Wei Wang Yanru Zhang Han Yang Xuming Wen 30 8 0 09 Oct 2023
DT-NeRF: Decomposed Triplane-Hash Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis Yaoyu Su Shaohui Wang Haoqian Wang CVBM 25 2 0 14 Sep 2023
ReliTalk: Relightable Talking Portrait Generation from a Single Video Haonan Qiu Zhaoxi Chen Yuming Jiang Hang Zhou Xiangyu Fan Lei Yang Wayne Wu Ziwei Liu DiffM VGen 31 10 0 05 Sep 2023
Text-to-Video: a Two-stage Framework for Zero-shot Identity-agnostic Talking-head Generation Zhichao Wang M. Dai Keld Lundgaard VGen DiffM 43 2 0 12 Aug 2023
Audio-driven Talking Face Generation with Stabilized Synchronization Loss Dogucan Yaman Fevziye Irem Eyiokur Leonard Barmann H. K. Ekenel Alexander Waibel CVBM 37 3 0 18 Jul 2023
Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis Jiahe Li Jiawei Zhang Xiao Bai Jun Zhou L. Gu 3DH 26 62 0 18 Jul 2023
EyeBAG: Accurate Control of Eye Blink and Gaze Based on Data Augmentation Leveraging Style Mixing Bryan S. Kim Jeong Young Jeong Wonjong Ryu CVBM 19 0 0 30 Jun 2023
Instruct-NeuralTalker: Editing Audio-Driven Talking Radiance Fields with Instructions Yuqi Sun Reian He Weimin Tan Bo Yan DiffM 30 2 0 19 Jun 2023
Ada-TTA: Towards Adaptive High-Quality Text-to-Talking Avatar Synthesis Zhe Ye Ziyue Jiang Yi Ren Jinglin Liu Chen Zhang Xiang Yin Zejun Ma Zhou Zhao 45 4 0 06 Jun 2023
Interactive Segment Anything NeRF with Feature Imitation Xiaokang Chen Jiaxiang Tang Diwen Wan Jingbo Wang Gang Zeng 51 22 0 25 May 2023
Uncovering and Quantifying Social Biases in Code Generation Yong-Jin Liu Xiaokang Chen Yan Gao Zhe Su Fengji Zhang Daoguang Zan Jian-Guang Lou Pin-Yu Chen Tsung-Yi Ho 36 19 0 24 May 2023
StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based Generator Jiazhi Guan Zhanwang Zhang Hang Zhou Tianshu Hu Kaisiyuan Wang ... Haocheng Feng Jingtuo Liu Errui Ding Ziwei Liu Jingdong Wang 42 57 0 09 May 2023
GeneFace++: Generalized and Stable Real-Time Audio-Driven 3D Talking Face Generation Zhenhui Ye Jinzheng He Ziyue Jiang Rongjie Huang Jia-Bin Huang Jinglin Liu Yixiang Ren Xiang Yin Zejun Ma Zhou Zhao CVBM 49 29 0 01 May 2023
Real-time 3D Semantic Scene Completion Via Feature Aggregation and Conditioned Prediction Xiaokang Chen Yajie Xing Gang Zeng 3DPC 29 15 0 20 Mar 2023
Memories are One-to-Many Mapping Alleviators in Talking Face Generation Anni Tang Tianyu He Xuejiao Tan Jun Ling Liang Song CVBM 26 23 0 09 Dec 2022
Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video Xuan Gao Chenglai Zhong Jun Xiang Yang Hong Yudong Guo Juyong Zhang 3DH CVBM 69 128 0 12 Oct 2022
DeVRF: Fast Deformable Voxel Radiance Fields for Dynamic Scenes Jia-Wei Liu Yan-Pei Cao Weijia Mao Wenqiao Zhang David Junhao Zhang Jussi Keppo Ying Shan Xiaohu Qie Mike Zheng Shou 169 112 0 31 May 2022
Fast Dynamic Radiance Fields with Time-Aware Neural Voxels Jiemin Fang Taoran Yi Xinggang Wang Lingxi Xie Xiaopeng Zhang Wenyu Liu Matthias Nießner Qi Tian VGen 155 290 0 30 May 2022
EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model Xinya Ji Hang Zhou Kaisiyuan Wang Qianyi Wu Wayne Wu Feng Xu Xun Cao CVBM 54 157 0 30 May 2022