Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration

19 October 2020

Shuang Li

Sanja Fidler

Antonio Torralba

LM&Ro

ArXiv PDF HTML

Papers citing "Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration"

36 / 36 papers shown

Title
R^3-VQA: "Read the Room" by Video Social Reasoning Lixing Niu Jiapeng Li Xingping Yu Shu Wang Ruining Feng Bo Wu Ping Wei Y. Wang Lifeng Fan 45 0 0 07 May 2025
Generative AI in Embodied Systems: System-Level Analysis of Performance, Efficiency and Scalability Zishen Wan Jiayi Qian Yuhang Du Jason J. Jabbour Yilun Du Yang Katie Zhao A. Raychowdhury Tushar Krishna Vijay Janapa Reddi LM&Ro 91 0 0 26 Apr 2025
Auto-SLURP: A Benchmark Dataset for Evaluating Multi-Agent Frameworks in Smart Personal Assistant Lei Shen Xiaoyu Shen 56 0 0 25 Apr 2025
Few-Shot Task Learning through Inverse Generative Modeling Aviv Netanyahu Yilun Du Antonia Bronars Jyothish Pari J. Tenenbaum Tianmin Shu Pulkit Agrawal 49 1 0 07 Nov 2024
CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation Jie Liu Pan Zhou Yingjun Du Ah-Hwee Tan Cees G. M. Snoek J. Sonke E. Gavves LLMAG 29 2 0 07 Nov 2024
Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling Jinghan Li Zhicheng Sun Fei Li 96 1 0 02 Oct 2024
ET-Plan-Bench: Embodied Task-level Planning Benchmark Towards Spatial-Temporal Cognition with Foundation Models Lingfeng Zhang Yuening Wang Hongjian Gu Atia Hamidizadeh Zhanguang Zhang ... Tongtong Cao Yuzheng Zhuang Yingxue Zhang Jianye Hao Jianye Hao LM&Ro 39 1 0 02 Oct 2024
COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Models Kehui Liu Zixin Tang Dong Wang Z. Wang Bin Zhao Bin Zhao 38 11 0 23 Sep 2024
MuMA-ToM: Multi-modal Multi-Agent Theory of Mind Haojun Shi Suyu Ye Xinyu Fang Chuanyang Jin Leyla Isik Yen-Ling Kuo Tianmin Shu LLMAG 65 7 0 22 Aug 2024
Effectively Leveraging CLIP for Generating Situational Summaries of Images and Videos Dhruv Verma Debaditya Roy Basura Fernando 27 1 0 30 Jul 2024
COMBO: Compositional World Models for Embodied Multi-Agent Cooperation Hongxin Zhang Zeyuan Wang Qiushi Lyu Zheyuan Zhang Sunli Chen Tianmin Shu Yilun Du Kwonjoon Lee Yilun Du Chuang Gan 45 12 0 16 Apr 2024
Smart Help: Strategic Opponent Modeling for Proactive and Adaptive Robot Assistance in Households Zhihao Cao Zidong Wang Siwen Xie Anji Liu Lifeng Fan 22 4 0 13 Apr 2024
A Survey on Large Language Model-Based Game Agents Sihao Hu Tiansheng Huang Gaowen Liu Ramana Rao Kompella Gaowen Liu Selim Furkan Tekin Yichang Xu Zachary Yahn Ling Liu LLMAG LM&Ro AI4CE LM&MA 69 49 0 02 Apr 2024
GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment Lance Ying Kunal Jha Shivam Aarya Joshua B. Tenenbaum Antonio Torralba Tianmin Shu 42 13 0 17 Mar 2024
EgoGen: An Egocentric Synthetic Data Generator Gen Li Kai Zhao Siwei Zhang X. Lyu Mihai Dusmanu Yan Zhang Marc Pollefeys Siyu Tang EgoV VGen 36 14 0 16 Jan 2024
LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models Saaket Agashe Yue Fan Anthony Reyna Xin Eric Wang LLMAG LRM 97 10 0 05 Oct 2023
Predicting Routine Object Usage for Proactive Robot Assistance Maithili Patel Aswin Prakash Sonia Chernova AI4TS 29 8 0 12 Sep 2023
Building Cooperative Embodied Agents Modularly with Large Language Models Hongxin Zhang Weihua Du Jiaming Shan Qinhong Zhou Yilun Du J. Tenenbaum Tianmin Shu Chuang Gan LLMAG LM&Ro 50 157 0 05 Jul 2023
Language Models Meet World Models: Embodied Experiences Enhance Language Models Jiannan Xiang Tianhua Tao Yi Gu Tianmin Shu Zirui Wang Zichao Yang Zhiting Hu ALM LLMAG LM&Ro CLL 31 94 0 18 May 2023
Improving Human-Robot Collaboration via Computational Design Jixuan Zhi Jyh-Ming Lien 23 0 0 20 Mar 2023
Proactive Robot Assistance via Spatio-Temporal Object Modeling Maithili Patel Sonia Chernova 29 26 0 28 Nov 2022
Optimal Behavior Prior: Data-Efficient Human Models for Improved Human-AI Collaboration Mesut Yang Micah Carroll Anca Dragan 27 13 0 03 Nov 2022
Long-HOT: A Modular Hierarchical Approach for Long-Horizon Object Transport S. Narayanan Dinesh Jayaraman Manmohan Chandraker 16 1 0 28 Oct 2022
RCareWorld: A Human-centric Simulation World for Caregiving Robots Ruolin Ye Wenqiang Xu Haoyuan Fu Rajat Kumar Jenamani V. Nguyen Cewu Lu Katherine Dimitropoulou T. Bhattacharjee LM&Ro 19 39 0 19 Oct 2022
Learning Latent Traits for Simulated Cooperative Driving Tasks Jonathan A. DeCastro Deepak Gopinath Guy Rosman Emily S. Sumner Shabnam Hakimi Simon Stent 18 0 0 20 Jul 2022
HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers Yu-Wei Chao Chris Paxton Yu Xiang Wei Yang Balakumar Sundaralingam Tao Chen Adithyavairavan Murali Maya Cakmak D. Fox 62 17 0 19 May 2022
Learning Theory of Mind via Dynamic Traits Attribution D. Nguyen Phuoc Nguyen Hung Le Kien Do Svetha Venkatesh T. Tran 12 5 0 17 Apr 2022
Object Manipulation via Visual Target Localization Kiana Ehsani Ali Farhadi Aniruddha Kembhavi Roozbeh Mottaghi LM&Ro 24 9 0 15 Mar 2022
Pre-Trained Language Models for Interactive Decision-Making Shuang Li Xavier Puig Chris Paxton Yilun Du Clinton Jia Wang ... Anima Anandkumar Jacob Andreas Igor Mordatch Antonio Torralba Yuke Zhu LM&Ro 34 246 0 03 Feb 2022
Learning Two-Step Hybrid Policy for Graph-Based Interpretable Reinforcement Learning Tongzhou Mu Kaixiang Lin Fei Niu Govind Thattai OffRL 22 0 0 21 Jan 2022
Towards Controllable Agent in MOBA Games with Generative Modeling Shubao Zhang 32 0 0 15 Dec 2021
Agent-Centric Relation Graph for Object Visual Navigation X. Hu Youfang Lin Shuo Wang Zhihao Wu Kai Lv 31 18 0 29 Nov 2021
ToM2C: Target-oriented Multi-agent Communication and Cooperation with Theory of Mind Yuan-Fang Wang Fangwei Zhong Jing Xu Yizhou Wang LLMAG 19 68 0 15 Oct 2021
SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents Grgur Kovač Rémy Portelas Katja Hofmann Pierre-Yves Oudeyer ALM 27 6 0 02 Jul 2021
A Survey on Human-aware Robot Navigation Ronja Möller Antonino Furnari S. Battiato Aki Härmä G. Farinella 33 87 0 22 Jun 2021
Hierarchical Deep Temporal Models for Group Activity Recognition Mostafa S. Ibrahim S. Muralidharan Zhiwei Deng Arash Vahdat Greg Mori 77 445 0 09 Jul 2016