ING-VP: MLLMs cannot Play Easy Vision-based Games Yet

ING-VP: MLLMs cannot Play Easy Vision-based Games Yet

9 October 2024

Haoran Zhang

Wenhao Huang

Jiaheng Liu

Ge Zhang

ArXiv (abs)PDF HTML Github (9★)

Papers citing "ING-VP: MLLMs cannot Play Easy Vision-based Games Yet"

19 / 19 papers shown

Title
A Survey on Large Language Model-Based Game Agents Sihao Hu Tiansheng Huang Gaowen Liu Ramana Rao Kompella Gaowen Liu Selim Furkan Tekin Yichang Xu Zachary Yahn Ling Liu LLMAG LM&Ro AI4CE LM&MA 156 57 0 02 Apr 2024
Generative Multimodal Models are In-Context Learners Quan-Sen Sun Yufeng Cui Xiaosong Zhang Fan Zhang Qiying Yu ... Yueze Wang Yongming Rao Jingjing Liu Tiejun Huang Xinlong Wang MLLM LRM 149 283 0 20 Dec 2023
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI Xiang Yue Yuansheng Ni Kai Zhang Tianyu Zheng Ruoqi Liu ... Yibo Liu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen OSLM ELM VLM 234 943 0 27 Nov 2023
DesignGPT: Multi-Agent Collaboration in Design Shiying Ding Xinyi Chen Yan Fang Wenrui Liu Yiwu Qiu Chunlei Chai LLMAG AI4CE 47 17 0 20 Nov 2023
Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplation Shenzhi Wang Chang Liu Zilong Zheng Siyuan Qi Shuo Chen Qisen Yang Andrew Zhao Chaofei Wang Shiji Song Gao Huang LLMAG 87 69 0 02 Oct 2023
RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models Zekun Wang Zhongyuan Peng Haoran Que Jiaheng Liu Wangchunshu Zhou ... Wanli Ouyang Ke Xu Wenhu Chen Jie Fu Junran Peng LLMAG 72 94 0 01 Oct 2023
SPRINT: Scalable Policy Pre-Training via Language Instruction Relabeling Jesse Zhang Karl Pertsch Jiahui Zhang Joseph J. Lim LM&Ro 88 17 0 20 Jun 2023
LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models Peng Xu Wenqi Shao Kaipeng Zhang Peng Gao Shuo Liu Meng Lei Fanqing Meng Siyuan Huang Yu Qiao Ping Luo ELM MLLM 74 170 0 15 Jun 2023
AlphaBlock: Embodied Finetuning for Vision-Language Reasoning in Robot Manipulation Chuhao Jin Wenhui Tan Jiange Yang Bei Liu Ruihua Song Limin Wang Jianlong Fu LM&Ro LRM 50 24 0 30 May 2023
CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society Ge Li Hasan Hammoud Hani Itani Dmitrii Khizbullin Guohao Li SyDa ALM 128 498 0 31 Mar 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 1.4K 14,631 0 15 Mar 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 429 4,563 0 30 Jan 2023
Large Language Models are Pretty Good Zero-Shot Video Game Bug Detectors Mohammad Reza Taesiri Finlay Macklon Yihe Wang Hengshuo Shen Cor-Paul Bezemer ELM LLMAG MLLM 81 13 0 05 Oct 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 416 3,585 0 29 Apr 2022
The Chess Transformer: Mastering Play using Generative Language Models David Noever Matt Ciolino Josh Kalin 52 38 0 02 Aug 2020
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models Bryan A. Plummer Liwei Wang Christopher M. Cervantes Juan C. Caicedo Julia Hockenmaier Svetlana Lazebnik 202 2,071 0 19 May 2015
VQA: Visual Question Answering Aishwarya Agrawal Jiasen Lu Stanislaw Antol Margaret Mitchell C. L. Zitnick Dhruv Batra Devi Parikh CoGe 211 5,497 0 03 May 2015
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 413 43,777 0 01 May 2014
The Arcade Learning Environment: An Evaluation Platform for General Agents Marc G. Bellemare Yavar Naddaf J. Veness Michael Bowling 120 3,020 0 19 Jul 2012