Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning

26 May 2025

Papers citing "Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning"

50 / 56 papers shown

Title
SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement Xinze Wang Zhiyong Yang Chao Feng Hongjin Lu Linjie Li Chung-Ching Lin Kevin Qinghong Lin Furong Huang Lijuan Wang OODD ReLM LRM VLM 122 12 0 10 Apr 2025
Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey Yansen Wang Shengqiong Wu Yize Zhang William Yang Wang Ziwei Liu Jiebo Luo Hao Fei LRM 124 23 0 16 Mar 2025
LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL Yingzhe Peng Gongrui Zhang Miaosen Zhang Zhiyuan You Jie Liu Qipeng Zhu Kai Yang Xingzhong Xu Xin Geng Xu Yang LRM ReLM 138 52 0 10 Mar 2025
Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models Wenxuan Huang Bohan Jia Zijie Zhai Shaosheng Cao Zheyu Ye Fei Zhao Zhe Xu Yao Hu Shaohui Lin MU OffRL LRM MLLM ReLM VLM 93 85 0 09 Mar 2025
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI Daya Guo Dejian Yang Haowei Zhang Junxiao Song ... Shiyu Wang S. Yu Shunfeng Zhou Shuting Pan S.S. Li ReLM VLM OffRL AI4TS LRM 249 1,503 0 22 Jan 2025
ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding Xingyu Fu Minqian Liu Zhengyuan Yang John Corring Yijuan Lu Jianwei Yang Dan Roth D. Florêncio Cha Zhang LRM 46 7 0 10 Jan 2025
Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark Yunzhuo Hao Jiawei Gu Huichen Will Wang Linjie Li Zhiyong Yang Lijuan Wang Yu Cheng LRM 57 30 0 10 Jan 2025
Large Language Monkeys: Scaling Inference Compute with Repeated Sampling Bradley Brown Jordan Juravsky Ryan Ehrlich Ronald Clark Quoc V. Le Christopher Ré Azalia Mirhoseini ALM LRM 165 278 0 03 Jan 2025
CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratings Shanghaoran Quan Jiaxi Yang Bowen Yu Jian Xu Dayiheng Liu ... Zeyu Cui Yang Fan Yanzhe Zhang Binyuan Hui Junyang Lin ALM ELM LRM 95 24 0 02 Jan 2025
OLA-VLM: Elevating Visual Perception in Multimodal LLMs with Auxiliary Embedding Distillation Jitesh Jain Zhengyuan Yang Humphrey Shi Jianfeng Gao Jianwei Yang MLLM 79 2 0 12 Dec 2024
MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale Jarvis Guo Tuney Zheng Yuelin Bai Bo Li Yubo Wang King Zhu Yizhi Li Graham Neubig Wenhu Chen Xiang Yue LRM 111 36 0 06 Dec 2024
GPT-4o System Card OpenAI OpenAI : Aaron Hurst Adam Lerer Adam P. Goucher ... Yuchen He Yuchen Zhang Yujia Jin Yunxing Dai Yury Malkov MLLM 126 750 0 25 Oct 2024
MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models Hang Hua Yunlong Tang Ziyun Zeng Liangliang Cao Zhengyuan Yang Hangfeng He Chenliang Xu Jiebo Luo VLM CoGe 55 13 0 13 Oct 2024
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters Charlie Snell Jaehoon Lee Kelvin Xu Aviral Kumar LRM 108 576 0 06 Aug 2024
MM-Vet v2: A Challenging Benchmark to Evaluate Large Multimodal Models for Integrated Capabilities Weihao Yu Zhengyuan Yang Linfeng Ren Linjie Li Jianfeng Wang Kevin Qinghong Lin Chung-Ching Lin Zicheng Liu Lijuan Wang Xinchao Wang VLM MLLM 79 21 0 01 Aug 2024
CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs Zirui Wang Mengzhou Xia Luxi He Howard Chen Yitao Liu ... Haotian Liu Sadhika Malladi Alexis Chevalier Sanjeev Arora Danqi Chen 42 55 0 26 Jun 2024
Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs Shengbang Tong Ellis L Brown Penghao Wu Sanghyun Woo Manoj Middepogu ... Xichen Pan Austin Wang Rob Fergus Yann LeCun Saining Xie 3DV MLLM 80 321 0 24 Jun 2024
Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models Yushi Hu Weijia Shi Xingyu Fu Dan Roth Mari Ostendorf Luke Zettlemoyer Noah A. Smith Ranjay Krishna LRM 65 57 0 13 Jun 2024
BLINK: Multimodal Large Language Models Can See but Not Perceive Xingyu Fu Yushi Hu Bangzheng Li Yu Feng Haoyu Wang Xudong Lin Dan Roth Noah A. Smith Wei-Chiu Ma Ranjay Krishna VLM LRM MLLM 50 133 0 18 Apr 2024
Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models Wenshan Wu Shaoguang Mao Yadong Zhang Yan Xia Li Dong Lei Cui Furu Wei LRM 64 25 0 04 Apr 2024
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code Naman Jain King Han Alex Gu Wen-Ding Li Fanjia Yan Tianjun Zhang Sida I. Wang Armando Solar-Lezama Koushik Sen Ion Stoica ELM 71 365 0 12 Mar 2024
Aligning Modalities in Vision Large Language Models via Preference Fine-tuning Yiyang Zhou Chenhang Cui Rafael Rafailov Chelsea Finn Huaxiu Yao VLM MLLM 77 104 0 18 Feb 2024
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models Zhihong Shao Peiyi Wang Qihao Zhu Runxin Xu Jun-Mei Song ... Haowei Zhang Mingchuan Zhang Yiming Li Yu-Huan Wu Daya Guo ReLM LRM 82 953 0 05 Feb 2024
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks Zhe Chen Jiannan Wu Wenhai Wang Weijie Su Guo Chen ... Bin Li Ping Luo Tong Lu Yu Qiao Jifeng Dai VLM MLLM 213 1,036 0 21 Dec 2023
RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback M. Steyvers Yuan Yao Haoye Zhang Taiwen He Yifeng Han ... Xinyue Hu Zhiyuan Liu Hai-Tao Zheng Maosong Sun Tat-Seng Chua MLLM VLM 159 198 0 01 Dec 2023
GPQA: A Graduate-Level Google-Proof Q&A Benchmark David Rein Betty Li Hou Asa Cooper Stickland Jackson Petty Richard Yuanzhe Pang Julien Dirani Julian Michael Samuel R. Bowman AI4MH ELM 70 627 0 20 Nov 2023
CogVLM: Visual Expert for Pretrained Language Models Weihan Wang Qingsong Lv Wenmeng Yu Wenyi Hong Ji Qi ... Bin Xu Juanzi Li Yuxiao Dong Ming Ding Jie Tang VLM MLLM 51 462 0 06 Nov 2023
The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision) Zhengyuan Yang Linjie Li Kevin Qinghong Lin Jianfeng Wang Chung-Ching Lin Nasim Shakouri Mahmoudabadi Lijuan Wang LM&MA 37 619 0 29 Sep 2023
Aligning Large Multimodal Models with Factually Augmented RLHF Zhiqing Sun Sheng Shen Shengcao Cao Haotian Liu Chunyuan Li ... Liangyan Gui Yu-Xiong Wang Yiming Yang Kurt Keutzer Trevor Darrell VLM 79 351 0 25 Sep 2023
DreamLLM: Synergistic Multimodal Comprehension and Creation Runpei Dong Chunrui Han Yuang Peng Zekun Qi Zheng Ge ... Hao-Ran Wei Xiangwen Kong Xiangyu Zhang Kaisheng Ma Li Yi MLLM 57 190 0 20 Sep 2023
Multimodal Foundation Models: From Specialists to General-Purpose Assistants Chunyuan Li Zhe Gan Zhengyuan Yang Jianwei Yang Linjie Li Lijuan Wang Jianfeng Gao MLLM 140 244 0 18 Sep 2023
Let's Verify Step by Step Hunter Lightman V. Kosaraju Yura Burda Harrison Edwards Bowen Baker Teddy Lee Jan Leike John Schulman Ilya Sutskever K. Cobbe ALM OffRL LRM 107 1,044 0 31 May 2023
Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings Daniel Philip Rose Vaishnavi Himakunthala Andy Ouyang Ryan He Alex Mei Yujie Lu Michael Stephen Saxon Chinmay Sonar Diba Mirza William Yang Wang LRM 86 43 0 03 May 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 349 4,607 0 17 Apr 2023
Multimodal Chain-of-Thought Reasoning in Language Models Zhuosheng Zhang Aston Zhang Mu Li Hai Zhao George Karypis Alexander J. Smola LRM 68 430 0 02 Feb 2023
Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning Zhuowan Li Xingrui Wang Elias Stengel-Eskin Adam Kortylewski Wufei Ma Benjamin Van Durme Max Planck Institute for Informatics OOD LRM 74 64 0 01 Dec 2022
ReCo: Region-Controlled Text-to-Image Generation Zhengyuan Yang Jianfeng Wang Zhe Gan Linjie Li Kevin Qinghong Lin ... Nan Duan Zicheng Liu Ce Liu Michael Zeng Lijuan Wang DiffM 80 146 0 23 Nov 2022
Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning Pan Lu Liang Qiu Kai-Wei Chang Ying Nian Wu Song-Chun Zhu Tanmay Rajpurohit Peter Clark Ashwin Kalyan ReLM LRM 114 285 0 29 Sep 2022
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark Ashwin Kalyan ELM ReLM LRM 230 1,188 0 20 Sep 2022
GIT: A Generative Image-to-text Transformer for Vision and Language Jianfeng Wang Zhengyuan Yang Xiaowei Hu Linjie Li Kevin Qinghong Lin Zhe Gan Zicheng Liu Ce Liu Lijuan Wang VLM 102 543 0 27 May 2022
CoCa: Contrastive Captioners are Image-Text Foundation Models Jiahui Yu Zirui Wang Vijay Vasudevan Legg Yeung Mojtaba Seyedhosseini Yonghui Wu VLM CLIP OffRL 127 1,286 0 04 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 288 3,458 0 29 Apr 2022
ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning Ahmed Masry Do Xuan Long J. Tan Shafiq Joty Enamul Hoque AIMat 79 617 0 19 Mar 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 123 865 0 07 Feb 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 594 9,009 0 28 Jan 2022
Scaling Up Vision-Language Pre-training for Image Captioning Xiaowei Hu Zhe Gan Jianfeng Wang Zhengyuan Yang Zicheng Liu Yumao Lu Lijuan Wang MLLM VLM 97 248 0 24 Nov 2021
UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Faisal Ahmed Zicheng Liu Yumao Lu Lijuan Wang 68 115 0 23 Nov 2021
Florence: A New Foundation Model for Computer Vision Lu Yuan Dongdong Chen Yi-Ling Chen Noel Codella Xiyang Dai ... Zhen Xiao Jianwei Yang Michael Zeng Luowei Zhou Pengchuan Zhang VLM 109 897 0 22 Nov 2021
IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning Pan Lu Liang Qiu Jiaqi Chen Tony Xia Yizhou Zhao Wei Zhang Zhou Yu Xiaodan Liang Song-Chun Zhu AIMat 68 197 0 25 Oct 2021
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision Zirui Wang Jiahui Yu Adams Wei Yu Zihang Dai Yulia Tsvetkov Yuan Cao VLM MLLM 101 789 0 24 Aug 2021