COLLAGE: Collaborative Human-Agent Interaction Generation using Hierarchical Latent Diffusion and Language Models

30 September 2024

Divyanshu Daiya

Damon Conover

Aniket Bera

DiffM

ArXiv PDF HTML

Papers citing "COLLAGE: Collaborative Human-Agent Interaction Generation using Hierarchical Latent Diffusion and Language Models"

43 / 43 papers shown

Title
AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agents Jieming Cui Tengyu Liu Nian Liu Yaodong Yang Yixin Zhu Siyuan Huang 78 23 0 19 Mar 2024
Scaling Up Dynamic Human-Scene Interaction Modeling Nan Jiang Zhiyuan Zhang Hongjie Li Xiaoxuan Ma Zan Wang Yixin Chen Tengyu Liu Yixin Zhu Siyuan Huang 62 60 0 13 Mar 2024
MotionScript: Natural Language Descriptions for Expressive 3D Human Motions Payam Jome Yazdian Eric Liu Li Cheng Angelica Lim Li Cheng Angelica Lim 60 10 0 19 Dec 2023
Interactive Humanoid: Online Full-Body Motion Reaction Synthesis with Social Affordance Canonicalization and Forecasting Yunze Liu Changxi Chen Li Yi 68 16 0 14 Dec 2023
LEMON: Learning 3D Human-Object Interaction Relation from 2D Images Yuhang Yang Wei Zhai Hongcheng Luo Yang Cao Zheng-Jun Zha 86 26 0 14 Dec 2023
EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation Wenyang Zhou Zhiyang Dou Zeyu Cao Zhouyingcheng Liao Jingbo Wang Wenjia Wang Yuan Liu Taku Komura Wenping Wang Lingjie Liu VGen 63 51 0 04 Dec 2023
CG-HOI: Contact-Guided 3D Human-Object Interaction Generation Christian Diller Angela Dai 86 66 0 27 Nov 2023
MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representations Heyuan Yao Zhenhua Song Yuyang Zhou Tenglong Ao Baoquan Chen Libin Liu 74 42 0 16 Oct 2023
OmniControl: Control Any Joint at Any Time for Human Motion Generation Yiming Xie Varun Jampani Lei Zhong Deqing Sun Huaizu Jiang DiffM 47 115 0 12 Oct 2023
Physically Plausible Full-Body Hand-Object Interaction Synthesis Jona Braun Sammy Christen Muhammed Kocabas Emre Aksan Otmar Hilliges 52 40 0 14 Sep 2023
Synthesizing Physically Plausible Human Motions in 3D Scenes Liang Pan Jingbo Wang Buzhen Huang Junyu Zhang Haofan Wang Xu Tang Yangang Wang 56 29 0 17 Aug 2023
MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators Yaqi Zhang Di Huang B. Liu Shixiang Tang Yan Lu Lu Chen Lei Bai Qi Chu Nenghai Yu Wanli Ouyang 117 103 0 19 Jun 2023
CAMS: CAnonicalized Manipulation Spaces for Category-Level Functional Hand-Object Manipulation Synthesis Juntian Zheng Qingyuan Zheng Lixing Fang Yun-Hai Liu Li Yi 68 46 0 25 Mar 2023
Task-Oriented Human-Object Interactions Generation with Implicit Neural Representations Quanzhou Li Jingbo Wang Chen Change Loy Bo Dai 85 28 0 23 Mar 2023
Affordance Diffusion: Synthesizing Hand-Object Interactions Yufei Ye Xueting Li Abhi Gupta Shalini De Mello Stan Birchfield Jiaming Song Shubham Tulsiani Sifei Liu DiffM 77 78 0 21 Mar 2023
Synthesizing Physical Character-Scene Interactions Mohamed Hassan Yunrong Guo Tingwu Wang Michael J. Black Sanja Fidler Xue Bin Peng 81 84 0 02 Feb 2023
IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactions Anindita Ghosh Rishabh Dabral Vladislav Golyanik Christian Theobalt P. Slusallek DiffM 113 94 0 14 Dec 2022
MultiAct: Long-Term 3D Human Motion Generation from Multiple Action Labels T. Lee Gyeongsik Moon Kyoung Mu Lee 50 45 0 12 Dec 2022
Executing your Commands via Motion Diffusion in Latent Space Xin Chen Biao Jiang Wen Liu Zilong Huang Bin-Bin Fu Tao Chen Jingyi Yu Gang Yu VGen DiffM 81 359 0 08 Dec 2022
Human Motion Diffusion Model Guy Tevet Sigal Raab Brian Gordon Yonatan Shafir Daniel Cohen-Or Amit H. Bermano DiffM VGen 260 756 0 29 Sep 2022
TEACH: Temporal Action Composition for 3D Humans Nikos Athanasiou Mathis Petrovich Michael J. Black Gül Varol 122 147 0 09 Sep 2022
FLAME: Free-form Language-based Motion Synthesis & Editing Jihoon Kim Jiseob Kim Sungjoon Choi VGen 79 210 0 01 Sep 2022
MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model Mingyuan Zhang Zhongang Cai Liang Pan Fangzhou Hong Xinying Guo Lei Yang Ziwei Liu DiffM VGen 94 568 0 31 Aug 2022
TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts Chuan Guo Xinxin Xuo Sen Wang Li Cheng VGen 113 239 0 04 Jul 2022
Learning to Use Chopsticks in Diverse Gripping Styles Zeshi Yang KangKang Yin Libin Liu 68 30 0 28 May 2022
Towards Diverse and Natural Scene-aware 3D Human Motion Synthesis Jingbo Wang Yu Rong Jingyuan Liu Sijie Yan Dahua Lin Bo Dai 97 85 0 25 May 2022
TEMOS: Generating diverse human motions from textual descriptions Mathis Petrovich Michael J. Black Gül Varol 95 386 0 25 Apr 2022
MotionCLIP: Exposing Human Motion Generation to CLIP Space Guy Tevet Brian Gordon Amir Hertz Amit H. Bermano Daniel Cohen-Or CLIP 110 341 0 15 Mar 2022
SAGA: Stochastic Whole-Body Grasping with Contact Yongpeng Wu Jiahao Wang Yan Zhang Siwei Zhang Otmar Hilliges Feng Yu Siyu Tang SLR 70 81 0 19 Dec 2021
CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation Y. Tashiro Jiaming Song Yang Song Stefano Ermon BDL DiffM 45 539 0 07 Jul 2021
Action-Conditioned 3D Human Motion Synthesis with Transformer VAE Mathis Petrovich Michael J. Black Gül Varol ViT 88 503 0 12 Apr 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 861 29,341 0 26 Feb 2021
Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting Kashif Rasul Calvin Seward Ingmar Schuster Roland Vollgraf DiffM 91 316 0 28 Jan 2021
Convolutional Autoencoders for Human Motion Infilling Manuel Kaufmann Emre Aksan Mingli Song Fabrizio Pece R. Ziegler Otmar Hilliges 3DH 51 101 0 22 Oct 2020
Denoising Diffusion Implicit Models Jiaming Song Chenlin Meng Stefano Ermon VLM DiffM 231 7,350 0 06 Oct 2020
Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wild Jason Y. Zhang Sam Pepose Hanbyul Joo Deva Ramanan Jitendra Malik Angjoo Kanazawa 105 148 0 30 Jul 2020
Action2Motion: Conditioned Generation of 3D Human Motions Chuan Guo Wei Ji Sen Wang Shihao Zou Qingyao Sun Annan Deng Minglun Gong Li Cheng 62 419 0 30 Jul 2020
Long-term Human Motion Prediction with Scene Context Zhe Cao Hang Gao K. Mangalam Qi-Zhi Cai Minh Vo Jitendra Malik 3DH 90 251 0 07 Jul 2020
Expressive Body Capture: 3D Hands, Face, and Body from a Single Image Georgios Pavlakos Vasileios Choutas N. Ghorbani Timo Bolkart Ahmed A. A. Osman Dimitrios Tzionas Michael J. Black 3DH 50 1,713 0 11 Apr 2019
Context-aware Human Motion Prediction Enric Corona Albert Pumarola Guillem Alenyà Francesc Moreno-Noguer 3DH 85 142 0 06 Apr 2019
Graph Attention Networks Petar Velickovic Guillem Cucurull Arantxa Casanova Adriana Romero Pietro Lio Yoshua Bengio GNN 459 20,124 0 30 Oct 2017
The KIT Motion-Language Dataset Matthias Plappert Christian Mandery Tamim Asfour 230 285 0 13 Jul 2016
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation Kyunghyun Cho B. V. Merrienboer Çağlar Gülçehre Dzmitry Bahdanau Fethi Bougares Holger Schwenk Yoshua Bengio AIMat 1.0K 23,338 0 03 Jun 2014