"Task Success" is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviors

6 February 2024

Papers citing ""Task Success" is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviors"

22 / 22 papers shown

Title
Evaluating Vision-Language Models as Evaluators in Path Planning Mohamed Aghzal Xiang Yue Erion Plaku Ziyu Yao LRM 165 1 0 27 Nov 2024
Semantically Safe Robot Manipulation: From Semantic Scene Understanding to Motion Safeguards Lukas Brunke Yanni Zhang Ralf Romer Jack Naimer Nikola Staykov Siqi Zhou Angela P. Schoellig 87 5 0 19 Oct 2024
To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning Zayne Sprague Fangcong Yin Juan Diego Rodriguez Dongwei Jiang Manya Wadhwa Prasann Singhal Xinyu Zhao Xi Ye Kyle Mahowald Greg Durrett ReLM LRM 207 130 0 18 Sep 2024
Self-Rewarding Language Models Weizhe Yuan Richard Yuanzhe Pang Kyunghyun Cho Xian Li Sainbayar Sukhbaatar Jing Xu Jason Weston ReLM SyDa ALM LRM 365 337 0 18 Jan 2024
ChatMPC: Natural Language based MPC Personalization Yuya Miyaoka Masaki Inoue Tomotaka Nii 66 4 0 12 Sep 2023
Language to Rewards for Robotic Skill Synthesis Wenhao Yu Nimrod Gileadi Chuyuan Fu Sean Kirmani Kuang-Huei Lee ... N. Heess Dorsa Sadigh Jie Tan Yuval Tassa F. Xia LM&Ro 90 282 0 14 Jun 2023
Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning L. Guan Karthik Valmeekam S. Sreedharan Subbarao Kambhampati LLMAG 72 176 0 24 May 2023
"No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomy Yuchen Cui Siddharth Karamcheti Raj Palleti Nidhya Shivakumar Percy Liang Dorsa Sadigh LM&Ro 89 82 0 06 Jan 2023
Constitutional AI: Harmlessness from AI Feedback Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell John Kernion ... Dario Amodei Nicholas Joseph Sam McCandlish Tom B. Brown Jared Kaplan SyDa MoMe 209 1,640 0 15 Dec 2022
Relative Behavioral Attributes: Filling the Gap between Symbolic Goal Specification and Reward Learning from Human Preferences L. Guan Karthik Valmeekam Subbarao Kambhampati 75 8 0 28 Oct 2022
LATTE: LAnguage Trajectory TransformEr A. Bucker Luis F. C. Figueredo Sami Haddadin Ashish Kapoor Shuang Ma Sai H. Vemprala Rogerio Bonatti LM&Ro 107 59 0 04 Aug 2022
Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation? Yuchen Cui S. Niekum Abhi Gupta Vikash Kumar Aravind Rajeswaran LM&Ro 86 79 0 23 Apr 2022
Correcting Robot Plans with Natural Language Feedback Pratyusha Sharma Balakumar Sundaralingam Valts Blukis Chris Paxton Tucker Hermans Antonio Torralba Jacob Andreas Dieter Fox 3DV LM&Ro 67 93 0 11 Apr 2022
Do As I Can, Not As I Say: Grounding Language in Robotic Affordances Michael Ahn Anthony Brohan Noah Brown Yevgen Chebotar Omar Cortes ... Ted Xiao Peng Xu Sichun Xu Mengyuan Yan Andy Zeng LM&Ro 192 1,984 0 04 Apr 2022
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language Andy Zeng Maria Attarian Brian Ichter K. Choromanski Adrian S. Wong ... Michael S. Ryoo Vikas Sindhwani Johnny Lee Vincent Vanhoucke Peter R. Florence ReLM LRM 158 588 0 01 Apr 2022
R3M: A Universal Visual Representation for Robot Manipulation Suraj Nair Aravind Rajeswaran Vikash Kumar Chelsea Finn Abhi Gupta LM&Ro 101 584 0 23 Mar 2022
DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection Hao Zhang Feng Li Shilong Liu Lei Zhang Hang Su Jun Zhu L. Ni H. Shum ViT 179 1,460 0 07 Mar 2022
The Surprising Effectiveness of Representation Learning for Visual Imitation Jyothish Pari Nur Muhammad (Mahi) Shafiullah Sridhar Pandian Arunachalam Lerrel Pinto SSL 99 171 0 02 Dec 2021
Visual Imitation Made Easy S. Young Dhiraj Gandhi Shubham Tulsiani Abhinav Gupta Pieter Abbeel Lerrel Pinto 79 140 0 11 Aug 2020
REALM: Retrieval-Augmented Language Model Pre-Training Kelvin Guu Kenton Lee Zora Tung Panupong Pasupat Ming-Wei Chang RALM 142 2,116 0 10 Feb 2020
Leveraging Human Guidance for Deep Reinforcement Learning Tasks Ruohan Zhang F. Torabi L. Guan D. Ballard Peter Stone 50 87 0 21 Sep 2019
Deep reinforcement learning from human preferences Paul Christiano Jan Leike Tom B. Brown Miljan Martic Shane Legg Dario Amodei 218 3,365 0 12 Jun 2017