Language Models as Black-Box Optimizers for Vision-Language Models

12 September 2023

Papers citing "Language Models as Black-Box Optimizers for Vision-Language Models"

21 / 21 papers shown

Title
Towards Understanding Camera Motions in Any Video Zhiqiu Lin Siyuan Cen Daniel Jiang Jay Karhade Hewei Wang ... Rushikesh Zawar Xue Bai Yilun Du Chuang Gan Deva Ramanan VGen 32 0 0 21 Apr 2025
ProAPO: Progressively Automatic Prompt Optimization for Visual Classification Xiangyan Qu Gaopeng Gou Jiamin Zhuang Jing Yu Kun Song Qihao Wang Yili Li Gang Xiong VLM 91 0 0 13 Mar 2025
GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Models Muhammad Jehanzeb Mirza Mengjie Zhao Zhuoyuan Mao Sivan Doveh Wei Lin ... Yuki Mitsufuji Horst Possegger Rogerio Feris Leonid Karlinsky James Glass VLM 84 1 0 08 Oct 2024
Automated Black-box Prompt Engineering for Personalized Text-to-Image Generation Yutong He Alexander Robey Naoki Murata Yiding Jiang J. Williams George Pappas Hamed Hassani Yuki Mitsufuji Ruslan Salakhutdinov J. Zico Kolter DiffM 104 4 0 28 Mar 2024
Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models Elaine Sui Xiaohan Wang Serena Yeung-Levy VLM 30 5 0 19 Mar 2024
BlackVIP: Black-Box Visual Prompting for Robust Transfer Learning Changdae Oh Hyeji Hwang Hee-young Lee Yongtaek Lim Geunyoung Jung Jiyoung Jung Hosik Choi Kyungwoo Song VLM VPVLM 85 57 0 26 Mar 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 298 2,232 0 22 Mar 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 278 4,244 0 30 Jan 2023
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models Manli Shu Weili Nie De-An Huang Zhiding Yu Tom Goldstein Anima Anandkumar Chaowei Xiao VLM VPVLM 186 282 0 15 Sep 2022
What does a platypus look like? Generating customized prompts for zero-shot image classification Sarah M Pratt Ian Covert Rosanne Liu Ali Farhadi VLM 131 212 0 07 Sep 2022
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 328 4,077 0 24 May 2022
BBTv2: Towards a Gradient-Free Future with Large Language Models Tianxiang Sun Zhengfu He Hong Qian Yunhua Zhou Xuanjing Huang Xipeng Qiu 108 53 0 23 May 2022
Instruction Induction: From Few Examples to Natural Language Task Descriptions Or Honovich Uri Shaham Samuel R. Bowman Omer Levy ELM LRM 120 136 0 22 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 319 11,953 0 04 Mar 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 392 4,137 0 28 Jan 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 385 8,495 0 28 Jan 2022
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 238 806 0 14 Oct 2021
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 348 2,271 0 02 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 280 3,848 0 18 Apr 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 310 3,708 0 11 Feb 2021
A Survey on Bias and Fairness in Machine Learning Ninareh Mehrabi Fred Morstatter N. Saxena Kristina Lerman Aram Galstyan SyDa FaML 323 4,212 0 23 Aug 2019