InstructAvatar: Text-Guided Emotion and Motion Control for Avatar Generation

24 May 2024

Yuchi Wang

Junliang Guo

Jianhong Bai

Runyi Yu

Tianyu He

Xu Tan

Xu Sun

Jiang Bian

DiffM

ArXiv (abs)PDF HTML

Papers citing "InstructAvatar: Text-Guided Emotion and Motion Control for Avatar Generation"

25 / 25 papers shown

Title
EmojiDiff: Advanced Facial Expression Control with High Identity Preservation in Portrait Generation Liangwei Jiang Ruida Li Zhifeng Zhang Shuo Fang Chenguang Ma DiffM 142 1 0 02 Dec 2024
Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance Qingcheng Zhao Pengyu Long Qixuan Zhang Dafei Qin Hanming Liang Longwen Zhang Yingliang Zhang Jingyi Yu Lan Xu DiffM 3DH 77 26 0 28 Jan 2024
VividTalk: One-Shot Audio-Driven Talking Head Generation Based on 3D Hybrid Prior Xusen Sun Longhao Zhang Hao Zhu Peng Zhang Bang Zhang Xinya Ji Kangneng Zhou Daiheng Gao Liefeng Bo Xun Cao VGen 93 27 0 04 Dec 2023
Collaborative Diffusion for Multi-Modal Face Generation and Editing Ziqi Huang Kelvin C. K. Chan Yuming Jiang Ziwei Liu DiffM 69 111 0 20 Apr 2023
DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance Longwen Zhang Qiwei Qiu Hongyang Lin Qixuan Zhang Cheng Shi Wei Yang Ye Shi Sibei Yang Lan Xu Jingyi Yu 3DH 88 78 0 01 Apr 2023
Adding Conditional Control to Text-to-Image Diffusion Models Lvmin Zhang Anyi Rao Maneesh Agrawala AI4CE 182 4,168 1 10 Feb 2023
MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptation Bo Zhang Chenyang Qi Pan Zhang Bo Zhang Hsiang-Tao Wu Dong Chen Qifeng Chen Yong Wang Fang Wen 78 56 0 15 Dec 2022
Audio-Driven Co-Speech Gesture Video Generation Xian Liu Qianyi Wu Hang Zhou Yuanqi Du Wayne Wu Dahua Lin Ziwei Liu SLR VGen 90 51 0 05 Dec 2022
SPACE: Speech-driven Portrait Animation with Controllable Expression Francesco Ferroni Arun Mallya Ting-Chun Wang Rafael Valle Xuan Li VGen 68 47 0 17 Nov 2022
EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model Xinya Ji Hang Zhou Kaisiyuan Wang Qianyi Wu Wayne Wu Feng Xu Xun Cao CVBM 114 164 0 30 May 2022
Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition Cheng Luo Siyang Song Weicheng Xie Linlin Shen Hatice Gunes CVBM 62 131 0 02 May 2022
Text and Image Guided 3D Avatar Generation and Manipulation Zehranaz Canfes M. Atasoy Alara Dirik Pinar Yanardag 3DH 98 45 0 12 Feb 2022
One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning Suzhe Wang Lincheng Li Yueqing Ding Xin Yu CVBM 101 119 0 06 Dec 2021
CLIP-Adapter: Better Vision-Language Models with Feature Adapters Peng Gao Shijie Geng Renrui Zhang Teli Ma Rongyao Fang Yongfeng Zhang Hongsheng Li Yu Qiao VLM CLIP 309 1,045 0 09 Oct 2021
PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering Yurui Ren Gezhong Li Yuanqi Chen Thomas H. Li Shan Liu DiffM VGen 113 227 0 17 Sep 2021
Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion Suzhe Wang Lincheng Li Yu-qiong Ding Changjie Fan Xin Yu VGen 98 165 0 20 Jul 2021
Towards Measuring Fairness in AI: the Casual Conversations Dataset C. Hazirbas Joanna Bitton Brian Dolhansky Jacqueline Pan Albert Gordo Cristian Canton Ferrer EGVM 81 93 0 06 Apr 2021
StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery Or Patashnik Zongze Wu Eli Shechtman Daniel Cohen-Or Dani Lischinski CLIP VLM 129 1,209 0 31 Mar 2021
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing Ting-Chun Wang Arun Mallya Xuan Li 3DH 113 486 0 30 Nov 2020
Denoising Diffusion Implicit Models Jiaming Song Chenlin Meng Stefano Ermon VLM DiffM 289 7,469 0 06 Oct 2020
Real Time Speech Enhancement in the Waveform Domain Alexandre Défossez Gabriel Synnaeve Yossi Adi 81 464 0 23 Jun 2020
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations Alexei Baevski Henry Zhou Abdel-rahman Mohamed Michael Auli SSL 297 5,837 0 20 Jun 2020
Conformer: Convolution-augmented Transformer for Speech Recognition Anmol Gulati James Qin Chung-Cheng Chiu Niki Parmar Yu Zhang ... Wei Han Shibo Wang Zhengdong Zhang Yonghui Wu Ruoming Pang 229 3,155 0 16 May 2020
MakeItTalk: Speaker-Aware Talking-Head Animation Yang Zhou Xintong Han Eli Shechtman J. Echevarria E. Kalogerakis Dingzeyu Li 68 423 0 27 Apr 2020
First Order Motion Model for Image Animation Aliaksandr Siarohin Stéphane Lathuilière Sergey Tulyakov Elisa Ricci N. Sebe VGen DiffM 112 934 0 29 Feb 2020