Robotic Environmental State Recognition with Pre-Trained Vision-Language
Models and Black-Box Optimization

Robotic Environmental State Recognition with Pre-Trained Vision-Language Models and Black-Box Optimization

26 September 2024

Kento Kawaharazuka

Yoshiki Obinata

Naoaki Kanazawa

Kei Okada

ArXiv (abs)PDF HTML

Papers citing "Robotic Environmental State Recognition with Pre-Trained Vision-Language Models and Black-Box Optimization"

16 / 16 papers shown

Title
Continuous Object State Recognition for Cooking Robots Using Pre-Trained Vision-Language Models and Black-box Optimization Kento Kawaharazuka Naoaki Kanazawa Yoshiki Obinata K. Okada Masayuki Inaba 62 6 0 13 Mar 2024
Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planning Dhruv Shah Michael Equi B. Osinski Fei Xia Brian Ichter Sergey Levine 3DV LM&Ro 81 102 0 16 Oct 2023
Recognition of Heat-Induced Food State Changes by Time-Series Use of Vision-Language Model for Cooking Robot Naoaki Kanazawa Kento Kawaharazuka Yoshiki Obinata K. Okada Masayuki Inaba LM&Ro 43 6 0 04 Sep 2023
VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models Wenlong Huang Chen Wang Ruohan Zhang Yunzhu Li Jiajun Wu Li Fei-Fei LM&Ro 127 518 0 12 Jul 2023
REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction Zeyi Liu Arpit Bahety Shuran Song LRM 108 127 0 27 Jun 2023
ImageBind: One Embedding Space To Bind Them All Rohit Girdhar Alaaeldin El-Nouby Zhuang Liu Mannat Singh Kalyan Vasudev Alwala Armand Joulin Ishan Misra VLM 164 941 0 09 May 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.5K 14,748 0 15 Mar 2023
Robotic Applications of Pre-Trained Vision-Language Models to Various Recognition Behaviors Kento Kawaharazuka Yoshiki Obinata Naoaki Kanazawa K. Okada Masayuki Inaba LM&Ro 62 12 0 10 Mar 2023
VQA-based Robotic State Recognition Optimized with Genetic Algorithm Kento Kawaharazuka Yoshiki Obinata Naoaki Kanazawa K. Okada Masayuki Inaba 38 16 0 09 Mar 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 432 4,656 0 30 Jan 2023
Visual Language Maps for Robot Navigation Chen Huang Oier Mees Andy Zeng Wolfram Burgard LM&Ro 243 369 0 11 Oct 2022
CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memory Nur Muhammad (Mahi) Shafiullah Chris Paxton Lerrel Pinto Soumith Chintala Arthur Szlam VLM LM&Ro CLIP 161 166 0 11 Oct 2022
Vision-Language Intelligence: Tasks, Representation Learning, and Large Models Feng Li Hao Zhang Yi-Fan Zhang Shixuan Liu Jian Guo L. Ni Pengchuan Zhang Lei Zhang AI4TS VLM 55 37 0 03 Mar 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 157 880 0 07 Feb 2022
CLIPort: What and Where Pathways for Robotic Manipulation Mohit Shridhar Lucas Manuelli Dieter Fox LM&Ro 123 661 0 24 Sep 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 993 29,871 0 26 Feb 2021