Inference Compute-Optimal Video Vision Language Models

24 May 2025

Papers citing "Inference Compute-Optimal Video Vision Language Models"

29 / 29 papers shown

Title
Large Language Monkeys: Scaling Inference Compute with Repeated Sampling Bradley Brown Jordan Juravsky Ryan Ehrlich Ronald Clark Quoc V. Le Christopher Ré Azalia Mirhoseini ALM LRM 218 299 0 03 Jan 2025
Exploring the Design Space of Visual Context Representation in Video MLLMs Yifan Du Yuqi Huo K. Zhou Zijia Zhao Haoyu Lu Han Huang Wayne Xin Zhao Bin Wang Weipeng Chen Ji-Rong Wen 45 2 0 17 Oct 2024
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 128 32 0 04 Oct 2024
Video Instruction Tuning With Synthetic Data Yuanhan Zhang Jinming Wu Wei Li Bo Li Zejun Ma Ziwei Liu Chunyuan Li SyDa VGen 99 192 0 03 Oct 2024
LLaVA-OneVision: Easy Visual Task Transfer Bo Li Yuanhan Zhang Dong Guo Renrui Zhang Feng Li Hao Zhang Kaichen Zhang Yanwei Li Ziwei Liu Chunyuan Li MLLM SyDa VLM 110 775 0 06 Aug 2024
LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding Haoning Wu Dongxu Li Bei Chen Junnan Li 89 148 0 22 Jul 2024
VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding Muhammad Maaz H. Rasheed Salman Khan Fahad A Khan VLM MLLM 70 58 0 13 Jun 2024
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis Chaoyou Fu Yuhan Dai Yondong Luo Lei Li Shuhuai Ren ... Xiawu Zheng Enhong Chen Caifeng Shan Xing Sun Xing Sun VLM MLLM 141 379 0 31 May 2024
Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward Ruohong Zhang Liangke Gui Zhiqing Sun Yihao Feng Keyang Xu ... Di Fu Chunyuan Li Alexander G. Hauptmann Yonatan Bisk Yiming Yang MLLM 94 71 0 01 Apr 2024
Language models scale reliably with over-training and on downstream tasks S. Gadre Georgios Smyrnis Vaishaal Shankar Suchin Gururangan Mitchell Wortsman ... Y. Carmon Achal Dave Reinhard Heckel Niklas Muennighoff Ludwig Schmidt ALM ELM LRM 126 44 0 13 Mar 2024
When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method Biao Zhang Zhongtao Liu Colin Cherry Orhan Firat LRM 97 145 0 27 Feb 2024
How predictable is language model benchmark performance? David Owen ELM LRM 45 19 0 09 Jan 2024
Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws Nikhil Sardana Jacob P. Portes Sasha Doubov Jonathan Frankle LRM 299 84 0 31 Dec 2023
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark Kunchang Li Yali Wang Yinan He Yizhuo Li Yi Wang ... Jilan Xu Guo Chen Ping Luo Limin Wang Yu Qiao VLM MLLM 139 470 0 28 Nov 2023
Perception Test: A Diagnostic Benchmark for Multimodal Video Models Viorica Puatruaucean Lucas Smaira Ankush Gupta Adrià Recasens Continente L. Markeeva ... Y. Aytar Simon Osindero Dima Damen Andrew Zisserman João Carreira VLM 161 167 0 23 May 2023
Getting ViT in Shape: Scaling Laws for Compute-Optimal Model Design Ibrahim Alabdulmohsin Xiaohua Zhai Alexander Kolesnikov Lucas Beyer VLM 97 63 0 22 May 2023
Fast Inference from Transformers via Speculative Decoding Yaniv Leviathan Matan Kalman Yossi Matias LRM 133 702 0 30 Nov 2022
Training Compute-Optimal Large Language Models Jordan Hoffmann Sebastian Borgeaud A. Mensch Elena Buchatskaya Trevor Cai ... Karen Simonyan Erich Elsen Jack W. Rae Oriol Vinyals Laurent Sifre AI4TS 200 1,946 0 29 Mar 2022
Finetuned Language Models Are Zero-Shot Learners Jason W. Wei Maarten Bosma Vincent Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai Quoc V. Le ALM UQCV 190 3,743 0 03 Sep 2021
NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions Junbin Xiao Xindi Shang Angela Yao Tat-Seng Chua 84 485 0 18 May 2021
Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval Max Bain Arsha Nagrani Gül Varol Andrew Zisserman VGen 136 1,173 0 01 Apr 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 918 29,372 0 26 Feb 2021
Scaling Laws for Transfer Danny Hernandez Jared Kaplan T. Henighan Sam McCandlish 75 245 0 02 Feb 2021
Scaling Laws for Autoregressive Generative Modeling T. Henighan Jared Kaplan Mor Katz Mark Chen Christopher Hesse ... Nick Ryder Daniel M. Ziegler John Schulman Dario Amodei Sam McCandlish 90 422 0 28 Oct 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 602 4,801 0 23 Jan 2020
CLEVRER: CoLlision Events for Video REpresentation and Reasoning Kexin Yi Yuta Saito Yunzhu Li Pushmeet Kohli Jiajun Wu Antonio Torralba J. Tenenbaum NAI 115 473 0 03 Oct 2019
ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering Zhou Yu D. Xu Jun-chen Yu Ting Yu Zhou Zhao Yueting Zhuang Dacheng Tao 107 463 0 06 Jun 2019
The "something something" video database for learning and evaluating visual common sense Raghav Goyal Samira Ebrahimi Kahou Vincent Michalski Joanna Materzynska S. Westphal ... Moritz Mueller-Freitag F. Hoppe Christian Thurau Ingo Bax Roland Memisevic VLM 84 1,531 0 13 Jun 2017
The Kinetics Human Action Video Dataset W. Kay João Carreira Karen Simonyan Brian Zhang Chloe Hillier ... Tim Green T. Back Apostol Natsev Mustafa Suleyman Andrew Zisserman 250 3,802 0 19 May 2017