v1v2v3 (latest)

Pretrained Language Models as Visual Planners for Human Assistance

17 April 2023

Ruta Desai

Papers citing "Pretrained Language Models as Visual Planners for Human Assistance"

43 / 43 papers shown

Title
COMBO: Compositional World Models for Embodied Multi-Agent Cooperation Hongxin Zhang Zeyuan Wang Qiushi Lyu Zheyuan Zhang Sunli Chen Tianmin Shu Yilun Du Kwonjoon Lee Yilun Du Chuang Gan 135 17 0 16 Apr 2024
PDPP: Projected Diffusion for Procedure Planning in Instructional Videos Hanlin Wang Yilu Wu Sheng Guo Limin Wang VGen DiffM 165 31 0 26 Mar 2023
Action Dynamics Task Graphs for Learning Plannable Representations of Procedural Tasks Weichao Mao Ruta Desai Michael L. Iuzzolino Nitin Kamra 66 5 0 11 Jan 2023
InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challenges Guo Chen Sen Xing Zhe Chen Yi Wang Kunchang Li ... Hongjie Zhang Tong Lu Yali Wang Liming Wang Yu Qiao 69 49 0 17 Nov 2022
Retrospectives on the Embodied AI Workshop Matt Deitke Dhruv Batra Yonatan Bisk Tommaso Campari Angel X. Chang ... Jesse Thomason Alexander Toshev Joanne Truong Luca Weihs Jiajun Wu LM&Ro 97 51 0 13 Oct 2022
VIMA: General Robot Manipulation with Multimodal Prompts Yunfan Jiang Agrim Gupta Zichen Zhang Guanzhi Wang Yongqiang Dou Yanjun Chen Li Fei-Fei Anima Anandkumar Yuke Zhu Linxi Fan LM&Ro 111 355 0 06 Oct 2022
Learning State-Aware Visual Representations from Audible Interactions Himangi Mittal Pedro Morgado Unnat Jain Abhinav Gupta 113 24 0 27 Sep 2022
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models Ishika Singh Valts Blukis Arsalan Mousavian Ankit Goyal Danfei Xu Jonathan Tremblay Dieter Fox Jesse Thomason Animesh Garg LM&Ro LLMAG 175 657 0 22 Sep 2022
A Generalist Agent Scott E. Reed Konrad Zolna Emilio Parisotto Sergio Gomez Colmenarejo Alexander Novikov ... Yutian Chen R. Hadsell Oriol Vinyals Mahyar Bordbar Nando de Freitas LM&Ro LLMAG AI4CE 211 826 0 12 May 2022
P3IV: Probabilistic Procedure Planning from Instructional Videos with Weak Supervision Henghui Zhao Isma Hadji Nikita Dvornik Konstantinos G. Derpanis Richard P. Wildes Allan D. Jepson 78 46 0 04 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 418 3,610 0 29 Apr 2022
Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videos Shao-Wei Liu Subarna Tripathi Somdeb Majumdar Xiaolong Wang EgoV 93 97 0 04 Apr 2022
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances Michael Ahn Anthony Brohan Noah Brown Yevgen Chebotar Omar Cortes ... Ted Xiao Peng Xu Sichun Xu Mengyuan Yan Andy Zeng LM&Ro 195 1,988 0 04 Apr 2022
Pre-Trained Language Models for Interactive Decision-Making Shuang Li Xavier Puig Chris Paxton Yilun Du Clinton Jia Wang ... Anima Anandkumar Jacob Andreas Igor Mordatch Antonio Torralba Yuke Zhu LM&Ro 112 262 0 03 Feb 2022
Human Hands as Probes for Interactive Object Understanding Mohit Goyal Sahil Modi Rishabh Goyal Saurabh Gupta 68 48 0 16 Dec 2021
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 477 7,827 0 11 Nov 2021
Ego4D: Around the World in 3,000 Hours of Egocentric Video Kristen Grauman Andrew Westbury Eugene Byrne Zachary Chavis Antonino Furnari ... Mike Zheng Shou Antonio Torralba Lorenzo Torresani Mingfei Yan Jitendra Malik EgoV 410 1,114 0 13 Oct 2021
Procedure Planning in Instructional Videos via Contextual Modeling and Model-based Policy Learning Jing Bi Jiebo Luo Chenliang Xu 118 49 0 05 Oct 2021
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 313 582 0 28 Sep 2021
PlaTe: Visually-Grounded Planning with Transformers in Procedural Tasks Jiankai Sun De-An Huang Bo Lu Yunhui Liu Bolei Zhou Animesh Garg 56 56 0 10 Sep 2021
BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments S. Srivastava Chengshu Li Michael Lingelbach Roberto Martín-Martín Fei Xia ... Chenxi Liu Silvio Savarese H. Gweon Jiajun Wu Li Fei-Fei LM&Ro 247 168 0 06 Aug 2021
Cooperative Exploration for Multi-Agent Deep Reinforcement Learning Iou-Jen Liu Unnat Jain Raymond A. Yeh Alex Schwing 82 104 0 23 Jul 2021
Habitat 2.0: Training Home Assistants to Rearrange their Habitat Andrew Szot Alexander Clegg Eric Undersander Erik Wijmans Yili Zhao ... Z. Kira V. Koltun Jitendra Malik Manolis Savva Dhruv Batra LM&Ro 109 527 0 28 Jun 2021
Multimodal Few-Shot Learning with Frozen Language Models Maria Tsimpoukelli Jacob Menick Serkan Cabi S. M. Ali Eslami Oriol Vinyals Felix Hill MLLM 185 789 0 25 Jun 2021
Offline Reinforcement Learning as One Big Sequence Modeling Problem Michael Janner Qiyang Li Sergey Levine OffRL 158 686 0 03 Jun 2021
Anticipative Video Transformer Rohit Girdhar Kristen Grauman ViT 65 211 0 03 Jun 2021
A Survey of Embodied AI: From Simulators to Research Tasks Jiafei Duan Samson Yu Tangyao Li Huaiyu Zhu Cheston Tan LM&Ro 68 294 0 08 Mar 2021
MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigation Saim Wani Shivansh Patel Unnat Jain Angel X. Chang Manolis Savva 93 107 0 07 Dec 2020
Long-Term Anticipation of Activities with Cycle Consistency Yazan Abu Farha Qiuhong Ke Bernt Schiele Juergen Gall AI4TS 68 44 0 02 Sep 2020
AllenAct: A Framework for Embodied AI Research Luca Weihs Jordi Salvador Klemen Kotar Unnat Jain Kuo-Hao Zeng Roozbeh Mottaghi Aniruddha Kembhavi LM&Ro AI4CE 63 75 0 28 Aug 2020
Bridging the Imitation Gap by Adaptive Insubordination Luca Weihs Unnat Jain Iou-Jen Liu Jordi Salvador Svetlana Lazebnik Aniruddha Kembhavi Alex Schwing 74 36 0 23 Jul 2020
Understanding Human Hands in Contact at Internet Scale Dandan Shan Jiaqi Geng Michelle Shu David Fouhey 108 325 0 11 Jun 2020
EGO-TOPO: Environment Affordances from Egocentric Video Tushar Nagarajan Yanghao Li Christoph Feichtenhofer Kristen Grauman EgoV 131 124 0 14 Jan 2020
Are Transformers universal approximators of sequence-to-sequence functions? Chulhee Yun Srinadh Bhojanapalli A. S. Rawat Sashank J. Reddi Sanjiv Kumar 124 359 0 20 Dec 2019
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Jiasen Lu Dhruv Batra Devi Parikh Stefan Lee SSL VLM 255 3,699 0 06 Aug 2019
Procedure Planning in Instructional Videos C. Chang De-An Huang Danfei Xu Ehsan Adeli Li Fei-Fei Juan Carlos Niebles 77 103 0 02 Jul 2019
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips Antoine Miech Dimitri Zhukov Jean-Baptiste Alayrac Makarand Tapaswi Ivan Laptev Josef Sivic VGen 122 1,208 0 07 Jun 2019
Next-Active-Object prediction from Egocentric Videos Antonino Furnari Sebastiano Battiato Kristen Grauman G. Farinella EgoV 54 97 0 10 Apr 2019
Cross-task weakly supervised learning from instructional videos Dimitri Zhukov Jean-Baptiste Alayrac R. G. Cinbis David Fouhey Ivan Laptev Josef Sivic SSL 136 250 0 19 Mar 2019
When will you do what? - Anticipating Temporal Occurrences of Activities Yazan Abu Farha Alexander Richard Juergen Gall 76 192 0 03 Apr 2018
AI2-THOR: An Interactive 3D Environment for Visual AI Eric Kolve Roozbeh Mottaghi Winson Han Eli VanderBilt Luca Weihs ... Daniel Gordon Yuke Zhu Aniruddha Kembhavi Abhinav Gupta Ali Farhadi LM&Ro 86 1,111 0 14 Dec 2017
Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification Saining Xie Chen Sun Jonathan Huang Zhuowen Tu Kevin Patrick Murphy 3DH 155 1,333 0 13 Dec 2017
On the Properties of Neural Machine Translation: Encoder-Decoder Approaches Kyunghyun Cho B. V. Merrienboer Dzmitry Bahdanau Yoshua Bengio AI4CE AIMat 259 6,791 0 03 Sep 2014