Audio-Driven Co-Speech Gesture Video Generation

5 December 2022

Ziwei Liu

Papers citing "Audio-Driven Co-Speech Gesture Video Generation"

41 / 41 papers shown

Title
CyberHost: Taming Audio-driven Avatar Diffusion Model with Region Codebook Attention Gaojie Lin Jianwen Jiang Chao Liang Tianyun Zhong Jiaqi Yang Yanbo Zheng VGen DiffM 93 18 0 03 Sep 2024
Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motion Evonne Ng Hanbyul Joo Liwen Hu Hao Li Trevor Darrell Angjoo Kanazawa Shiry Ginosar VGen 45 94 0 18 Apr 2022
Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation Xian Liu Qianyi Wu Hang Zhou Yinghao Xu Rui Qian Xinyi Lin Xiaowei Zhou Wayne Wu Bo Dai Bolei Zhou SLR 67 105 0 24 Mar 2022
Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory Lian Siyao Weijiang Yu Tianpei Gu Chunze Lin Quan Wang Chao Qian Chen Change Loy Ziwei Liu SLR 121 192 0 24 Mar 2022
Freeform Body Motion Generation from Speech Jing-Fen Xu Wei Zhang Yalong Bai Qi-Biao Sun Tao Mei SLR 63 18 0 04 Mar 2022
Visual Sound Localization in the Wild by Cross-Modal Interference Erasing Xian Liu Rui Qian Hang Zhou Di Hu Weiyao Lin Ziwei Liu Bolei Zhou Xiaowei Zhou 44 25 0 13 Feb 2022
Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation Xian Liu Yinghao Xu Qianyi Wu Hang Zhou Wayne Wu Bolei Zhou VGen DiffM 3DH 71 142 0 19 Jan 2022
Speech Drives Templates: Co-Speech Gesture Synthesis with Learned Templates Shenhan Qian Zhi Tu Yihao Zhi Wen Liu Shenghua Gao SLR 45 75 0 18 Aug 2021
Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoders Jing Li Di Kang Wenjie Pei Xuefei Zhe Ying Zhang Zhenyu He Linchao Bao SLR 64 106 0 15 Aug 2021
Motion Representations for Articulated Animation Aliaksandr Siarohin Oliver J. Woodford Jian Ren Menglei Chai Sergey Tulyakov OCL 150 269 0 22 Apr 2021
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation Hang Zhou Yasheng Sun Wayne Wu Chen Change Loy Xiaogang Wang Ziwei Liu CVBM 104 366 0 22 Apr 2021
MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement Alexander Richard Michael Zollhoefer Yandong Wen Fernando de la Torre Yaser Sheikh CVBM 66 200 0 16 Apr 2021
Learning Speech-driven 3D Conversational Gestures from Video I. Habibie Weipeng Xu Dushyant Mehta Lingjie Liu Hans-Peter Seidel Gerard Pons-Moll Mohamed A. Elgharib Christian Theobalt SLR CVBM 3DH 69 110 0 13 Feb 2021
Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agents Uttaran Bhattacharya Nicholas Rewkowski A. Banerjee P. Guhan Aniket Bera Tianyi Zhou LM&Ro 55 153 0 26 Jan 2021
Speech Gesture Generation from the Trimodal Context of Text, Audio, and Speaker Identity Youngwoo Yoon Bok Cha Joo-Haeng Lee Minsu Jang Jaeyeon Lee Jaehong Kim Geehyuk Lee 44 283 0 04 Sep 2020
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild Prajwal K R Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar EGVM 101 777 0 23 Aug 2020
Monocular Expressive Body Regression through Body-Driven Attention Vasileios Choutas Georgios Pavlakos Timo Bolkart Dimitrios Tzionas Michael J. Black 3DH CVBM 84 240 0 20 Aug 2020
Face2Face: Real-time Face Capture and Reenactment of RGB Videos Justus Thies Michael Zollhöfer Marc Stamminger Christian Theobalt Matthias Nießner 3DH PICV CVBM 55 1,913 0 29 Jul 2020
Style Transfer for Co-Speech Gesture Animation: A Multi-Speaker Conditional-Mixture Approach Chaitanya Ahuja Dong Won Lee Y. Nakano Louis-Philippe Morency 43 104 0 24 Jul 2020
First Order Motion Model for Image Animation Aliaksandr Siarohin Stéphane Lathuilière Sergey Tulyakov Elisa Ricci N. Sebe VGen DiffM 81 925 0 29 Feb 2020
Gesticulator: A framework for semantically-aware speech-driven gesture generation Taras Kucherenko Patrik Jonell S. V. Waveren G. Henter Simon Alexanderson Iolanda Leite Hedvig Kjellström SLR 47 180 0 25 Jan 2020
Audio-visual Recognition of Overlapped speech for the LRS2 dataset Jianwei Yu Shi-Xiong Zhang Jian Wu Shahram Ghorbani Bo Wu Shiyin Kang Shansong Liu Xunying Liu Helen Meng Dong Yu 71 73 0 06 Jan 2020
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 493 42,407 0 03 Dec 2019
Lightweight and Efficient End-to-End Speech Recognition Using Low-Rank Transformer Genta Indra Winata Samuel Cahyawijaya Zhaojiang Lin Zihan Liu Pascale Fung 46 76 0 30 Oct 2019
Language2Pose: Natural Language Grounded Pose Forecasting Chaitanya Ahuja Louis-Philippe Morency 69 273 0 02 Jul 2019
Learning Individual Styles of Conversational Gesture Shiry Ginosar Amir Bar Gefen Kohavi Caroline Chan Andrew Owens Jitendra Malik SLR 45 332 0 10 Jun 2019
Few-Shot Adversarial Learning of Realistic Neural Talking Head Models Egor Zakharov Aliaksandra Shysheya Egor Burkov Victor Lempitsky 3DH 147 629 0 20 May 2019
Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Loss Lele Chen R. Maddox Z. Duan Chenliang Xu CVBM 68 398 0 09 May 2019
Capture, Learning, and Synthesis of 3D Speaking Styles Daniel Cudeiro Timo Bolkart Cassidy Laidlaw Anurag Ranjan Michael J. Black CVBM 3DH 91 343 0 08 May 2019
3D Guided Fine-Grained Face Manipulation Z. Geng Chen Cao Sergey Tulyakov CVBM 3DH 74 99 0 24 Feb 2019
Animating Arbitrary Objects via Deep Motion Transfer Aliaksandr Siarohin Stéphane Lathuilière Sergey Tulyakov Elisa Ricci N. Sebe 65 347 0 20 Dec 2018
OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields Zhe Cao Gines Hidalgo Tomas Simon S. Wei Yaser Sheikh 3DH CVBM 121 4,590 0 18 Dec 2018
Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robots Youngwoo Yoon Woo-Ri Ko Minsu Jang Jaeyeon Lee Jaehong Kim Geehyuk Lee SLR 49 231 0 30 Oct 2018
Everybody Dance Now Caroline Chan Shiry Ginosar Tinghui Zhou Alexei A. Efros 105 777 0 22 Aug 2018
X2Face: A network for controlling face generation by using images, audio, and pose codes Olivia Wiles A. Sophia Koepke Andrew Zisserman CVBM 83 415 0 27 Jul 2018
Synthesizing Images of Humans in Unseen Poses Guha Balakrishnan Amy Zhao Adrian Dalca F. Durand John Guttag GAN 3DH 48 314 0 20 Apr 2018
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling Shaojie Bai J. Zico Kolter V. Koltun DRL 86 4,806 0 04 Mar 2018
Neural Discrete Representation Learning Aaron van den Oord Oriol Vinyals Koray Kavukcuoglu BDL SSL OCL 226 5,008 0 02 Nov 2017
In Defense of the Triplet Loss for Person Re-Identification Alexander Hermans Lucas Beyer Bastian Leibe DML 78 3,205 0 22 Mar 2017
Perceptual Losses for Real-Time Style Transfer and Super-Resolution Justin Johnson Alexandre Alahi Li Fei-Fei SupR 232 10,247 0 27 Mar 2016
Adam: A Method for Stochastic Optimization Diederik P. Kingma Jimmy Ba ODL 1.8K 150,039 0 22 Dec 2014