Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation

24 March 2022

Papers citing "Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation"

50 / 53 papers shown

Title
PianoMotion10M: Dataset and Benchmark for Hand Motion Generation in Piano Performance Qijun Gan Song Wang Shengtao Wu Jianke Zhu 244 1 0 13 Jun 2024
CoCoGesture: Toward Coherent Co-speech 3D Gesture Generation in the Wild Xingqun Qi Hengyuan Zhang Yatian Wang J. Pan Chen Liu ... Qixun Zhang Shanghang Zhang Wenhan Luo Qifeng Liu Qi-fei Liu DiffM SLR 134 7 0 27 May 2024
MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models Zunnan Xu Yukang Lin Haonan Han Sicheng Yang Ronghui Li Yachao Zhang Xiu Li Mamba 119 26 0 14 Mar 2024
SpeechAct: Towards Generating Whole-body Motion from Speech Jinsong Zhang Minjie Zhu Yuxiang Zhang Yebin Liu Kun Li 92 0 0 29 Nov 2023
Visual Sound Localization in the Wild by Cross-Modal Interference Erasing Xian Liu Rui Qian Hang Zhou Di Hu Weiyao Lin Ziwei Liu Bolei Zhou Xiaowei Zhou 51 25 0 13 Feb 2022
Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation Xian Liu Yinghao Xu Qianyi Wu Hang Zhou Wayne Wu Bolei Zhou VGen DiffM 3DH 80 143 0 19 Jan 2022
Speech Drives Templates: Co-Speech Gesture Synthesis with Learned Templates Shenhan Qian Zhi Tu Yihao Zhi Wen Liu Shenghua Gao SLR 52 76 0 18 Aug 2021
Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoders Jing Li Di Kang Wenjie Pei Xuefei Zhe Ying Zhang Zhenyu He Linchao Bao SLR 75 106 0 15 Aug 2021
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation Hang Zhou Yasheng Sun Wayne Wu Chen Change Loy Xiaogang Wang Ziwei Liu CVBM 110 371 0 22 Apr 2021
Visually Informed Binaural Audio Generation without Binaural Audios Xudong Xu Hang Zhou Ziwei Liu Bo Dai Xiaogang Wang Dahua Lin DiffM 46 58 0 13 Apr 2021
Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation Yapeng Tian Di Hu Chenliang Xu ObjD 77 88 0 05 Apr 2021
Learning Speech-driven 3D Conversational Gestures from Video I. Habibie Weipeng Xu Dushyant Mehta Lingjie Liu Hans-Peter Seidel Gerard Pons-Moll Mohamed A. Elgharib Christian Theobalt SLR CVBM 3DH 80 110 0 13 Feb 2021
Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agents Uttaran Bhattacharya Nicholas Rewkowski A. Banerjee P. Guhan Aniket Bera Tianyi Zhou LM&Ro 72 153 0 26 Jan 2021
GlocalNet: Class-aware Long-term Human Motion Synthesis Neeraj Battan Yudhik Agrawal Veeravalli Saisooryarao Aman Goel Avinash Sharma 3DH 52 15 0 19 Dec 2020
Speech Gesture Generation from the Trimodal Context of Text, Audio, and Speaker Identity Youngwoo Yoon Bok Cha Joo-Haeng Lee Minsu Jang Jaeyeon Lee Jaehong Kim Geehyuk Lee 49 283 0 04 Sep 2020
A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild Prajwal K R Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar EGVM 103 787 0 23 Aug 2020
Monocular Expressive Body Regression through Body-Driven Attention Vasileios Choutas Georgios Pavlakos Timo Bolkart Dimitrios Tzionas Michael J. Black 3DH CVBM 94 240 0 20 Aug 2020
Style Transfer for Co-Speech Gesture Animation: A Multi-Speaker Conditional-Mixture Approach Chaitanya Ahuja Dong Won Lee Y. Nakano Louis-Philippe Morency 47 104 0 24 Jul 2020
Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing Yapeng Tian Dingzeyu Li Chenliang Xu 105 184 0 21 Jul 2020
Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation Hang Zhou Xudong Xu Dahua Lin Xiaogang Wang Ziwei Liu DiffM 78 83 0 20 Jul 2020
Dance Revolution: Long-Term Dance Generation with Music via Curriculum Learning Ruozi Huang Huang Hu Wei Wu Kei Sawada Mi Zhang Daxin Jiang 62 121 0 11 Jun 2020
Generative Tweening: Long-term Inbetweening of 3D Human Motions Yi Zhou Jingwan Lu Connelly Barnes Jimei Yang Sitao Xiang Hao li GAN 3DH 64 44 0 18 May 2020
What comprises a good talking-head video generation?: A Survey and Benchmark Lele Chen Guofeng Cui Ziyi Kou Haitian Zheng Chenliang Xu EGVM 48 59 0 07 May 2020
MakeItTalk: Speaker-Aware Talking-Head Animation Yang Zhou Xintong Han Eli Shechtman J. Echevarria E. Kalogerakis Dingzeyu Li 68 423 0 27 Apr 2020
Music Gesture for Visual Sound Separation Chuang Gan Deng Huang Hang Zhao J. Tenenbaum Antonio Torralba 95 205 0 20 Apr 2020
In defence of metric learning for speaker recognition Joon Son Chung Jaesung Huh Seongkyu Mun Minjae Lee Hee-Soo Heo Soyeon Choe Chiheon Ham Sung-Ye Jung Bong-Jin Lee Icksang Han 67 437 0 26 Mar 2020
Gesticulator: A framework for semantically-aware speech-driven gesture generation Taras Kucherenko Patrik Jonell S. V. Waveren G. Henter Simon Alexanderson Iolanda Leite Hedvig Kjellström SLR 52 180 0 25 Jan 2020
Audio-visual Recognition of Overlapped speech for the LRS2 dataset Jianwei Yu Shi-Xiong Zhang Jian Wu Shahram Ghorbani Bo Wu Shiyin Kang Shansong Liu Xunying Liu Helen Meng Dong Yu 78 73 0 06 Jan 2020
Lightweight and Efficient End-to-End Speech Recognition Using Low-Rank Transformer Genta Indra Winata Samuel Cahyawijaya Zhaojiang Lin Zihan Liu Pascale Fung 62 76 0 30 Oct 2019
Self-supervised Moving Vehicle Tracking with Stereo Sound Chuang Gan Hang Zhao Peihao Chen David D. Cox Antonio Torralba 48 147 0 25 Oct 2019
Structured Prediction Helps 3D Human Motion Modelling Emre Aksan Manuel Kaufmann Otmar Hilliges 3DH 84 191 0 20 Oct 2019
Deep High-Resolution Representation Learning for Visual Recognition Jingdong Wang Ke Sun Tianheng Cheng Borui Jiang Chaorui Deng ... Yadong Mu Mingkui Tan Xinggang Wang Wenyu Liu Bin Xiao 393 3,627 0 20 Aug 2019
Learning Trajectory Dependencies for Human Motion Prediction Wei Mao Miaomiao Liu Mathieu Salzmann Hongdong Li 3DH 72 437 0 15 Aug 2019
Language2Pose: Natural Language Grounded Pose Forecasting Chaitanya Ahuja Louis-Philippe Morency 77 275 0 02 Jul 2019
Learning Individual Styles of Conversational Gesture Shiry Ginosar Amir Bar Gefen Kohavi Caroline Chan Andrew Owens Jitendra Malik SLR 48 333 0 10 Jun 2019
Hierarchical Cross-Modal Talking Face Generationwith Dynamic Pixel-Wise Loss Lele Chen R. Maddox Z. Duan Chenliang Xu CVBM 83 400 0 09 May 2019
The Sound of Motions Hang Zhao Chuang Gan Wei-Chiu Ma Antonio Torralba 83 254 0 11 Apr 2019
OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields Zhe Cao Gines Hidalgo Tomas Simon S. Wei Yaser Sheikh 3DH CVBM 124 4,598 0 18 Dec 2018
Human Motion Prediction via Spatio-Temporal Inpainting Alejandro Hernandez Ruiz Juergen Gall Francesc Moreno-Noguer 3DH 64 212 0 13 Dec 2018
Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robots Youngwoo Yoon Woo-Ri Ko Minsu Jang Jaeyeon Lee Jaehong Kim Geehyuk Lee SLR 53 231 0 30 Oct 2018
Talking Face Generation by Adversarially Disentangled Audio-Visual Representation Hang Zhou Yu Liu Ziwei Liu Ping Luo Xiaogang Wang CVBM 92 443 0 20 Jul 2018
Convolutional Sequence to Sequence Model for Human Dynamics Chen Li Zhen Zhang Wee Sun Lee Gim Hee Lee 3DH 65 327 0 02 May 2018
Neural Kinematic Networks for Unsupervised Motion Retargetting Ruben Villegas Jimei Yang Duygu Ceylan Honglak Lee GAN 84 220 0 16 Apr 2018
The Sound of Pixels Hang Zhao Chuang Gan Andrew Rouditchenko Carl Vondrick Josh H. McDermott Antonio Torralba VLM 102 536 0 09 Apr 2018
Audio-Visual Event Localization in Unconstrained Videos Yapeng Tian Jing Shi Bochen Li Zhiyao Duan Chenliang Xu 101 439 0 23 Mar 2018
An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling Shaojie Bai J. Zico Kolter V. Koltun DRL 97 4,845 0 04 Mar 2018
HP-GAN: Probabilistic 3D human motion prediction via GAN Emad Barsoum J. Kender Zicheng Liu 3DH 86 331 0 27 Nov 2017
Learning Human Motion Models for Long-term Predictions Partha Ghosh Mingli Song Emre Aksan Otmar Hilliges 3DH 62 240 0 10 Apr 2017
Feature Pyramid Networks for Object Detection Nayeon Lee Piotr Dollár Ross B. Girshick Kaiming He Bharath Hariharan Serge J. Belongie ObjD 483 22,143 0 09 Dec 2016
Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network Wenzhe Shi Jose Caballero Ferenc Huszár J. Totz Andrew P. Aitken Rob Bishop Daniel Rueckert Zehan Wang SupR 335 5,239 0 16 Sep 2016