ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models

19 April 2022

Jianwei Yang

Zicheng Liu

Papers citing "ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models"

50 / 111 papers shown

Title
Large Language Model Routing with Benchmark Datasets Tal Shnitzer Anthony Ou Mírian Silva Kate Soule Yuekai Sun Justin Solomon Neil Thompson Mikhail Yurochkin RALM 16 58 0 27 Sep 2023
MoCaE: Mixture of Calibrated Experts Significantly Improves Object Detection Kemal Oksuz Selim Kuzucu Tom Joy P. Dokania MoE 24 5 0 26 Sep 2023
TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance Kan Wu Houwen Peng Zhenghong Zhou Bin Xiao Mengchen Liu ... Xi Xi Chen Xinggang Wang Hongyang Chao Han Hu VLM OODD 29 53 0 21 Sep 2023
Hydra: Multi-head Low-rank Adaptation for Parameter Efficient Fine-tuning Sanghyeon Kim Hyunmo Yang Younghyun Kim Youngjoon Hong Eunbyung Park AI4CE 32 16 0 13 Sep 2023
Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehicle Maria Priisalu 35 1 0 12 Sep 2023
How to Evaluate the Generalization of Detection? A Benchmark for Comprehensive Open-Vocabulary Detection Yi Yao Peng Liu Tiancheng Zhao Qianqian Zhang Jiajia Liao Chunxin Fang Kyusong Lee Qing Wang VLM ObjD 29 12 0 25 Aug 2023
Benchmarking and Analyzing Generative Data for Visual Recognition Bo-wen Li Haotian Liu Liangyu Chen Yong Jae Lee C. Li Ziwei Liu EGVM VLM 18 4 0 25 Jul 2023
Described Object Detection: Liberating Object Detection with Flexible Expressions Chi Xie Zhao Zhang YiXuan Wu Feng Zhu Rui Zhao Shuang Liang ObjD 39 31 0 24 Jul 2023
VideoGLUE: Video General Understanding Evaluation of Foundation Models Liangzhe Yuan N. B. Gundavarapu Long Zhao Hao Zhou Huayu Chen ... Florian Schroff Hartwig Adam Ming Yang Ting Liu Boqing Gong ELM 40 9 0 06 Jul 2023
Visual Instruction Tuning with Polite Flamingo Delong Chen Jianfeng Liu Wenliang Dai Baoyuan Wang MLLM 34 42 0 03 Jul 2023
Benchmarking Zero-Shot Recognition with Vision-Language Models: Challenges on Granularity and Specificity Zhenlin Xu Yi Zhu Tiffany Deng Abhay Mittal Yanbei Chen Manchen Wang Paolo Favaro Joseph Tighe Davide Modolo VLM CoGe 25 8 0 28 Jun 2023
DesCo: Learning Object Recognition with Rich Language Descriptions Liunian Harold Li Zi-Yi Dou Nanyun Peng Kai-Wei Chang ObjD VLM 34 20 0 24 Jun 2023
Scaling Open-Vocabulary Object Detection Matthias Minderer A. Gritsenko N. Houlsby VLM ObjD 29 178 0 16 Jun 2023
Contrasting Intra-Modal and Ranking Cross-Modal Hard Negatives to Enhance Visio-Linguistic Compositional Understanding Le Zhang Rabiul Awal Aishwarya Agrawal CoGe VLM 31 9 0 15 Jun 2023
VISION Datasets: A Benchmark for Vision-based InduStrial InspectiON Haoping Bai Shancong Mou Tatiana Likhomanenko R. G. Cinbis Oncel Tuzel Ping Huang Jiulong Shan Jianjun Shi Mengsi Cao VLM 16 23 0 13 Jun 2023
Matte Anything: Interactive Natural Image Matting with Segment Anything Models J. Yao Xinggang Wang Lang Ye Wenyu Liu 28 38 0 07 Jun 2023
Enhancing CLIP with CLIP: Exploring Pseudolabeling for Limited-Label Prompt Tuning Cristina Menghini Andrew T. Delworth Stephen H. Bach VLM 15 21 0 02 Jun 2023
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day Chunyuan Li Cliff Wong Sheng Zhang Naoto Usuyama Haotian Liu Jianwei Yang Tristan Naumann Hoifung Poon Jianfeng Gao LM&MA MedIm 56 703 0 01 Jun 2023
Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Models Sivan Doveh Assaf Arbelle Sivan Harary Roei Herzig Donghyun Kim ... Yikang Shen Raja Giryes Rogerio Feris S. Ullman Leonid Karlinsky VLM CoGe 62 52 0 31 May 2023
Multi-modal Queried Object Detection in the Wild Yifan Xu Mengdan Zhang Chaoyou Fu Peixian Chen Xiaoshan Yang Ke Li Changsheng Xu ObjD VLM 35 30 0 30 May 2023
Mitigating Test-Time Bias for Fair Image Retrieval Fanjie Kong Shuai Yuan Weituo Hao Ricardo Henao 29 16 0 23 May 2023
Coarse-to-Fine Contrastive Learning in Image-Text-Graph Space for Improved Vision-Language Compositionality Harman Singh Pengchuan Zhang Qifan Wang Mengjiao MJ Wang Wenhan Xiong Jingfei Du Yu Chen CoGe VLM 29 24 0 23 May 2023
On the Hidden Mystery of OCR in Large Multimodal Models Yuliang Liu Zhang Li Mingxin Huang Chunyuan Li Dezhi Peng Mingyu Liu Lianwen Jin Xiang Bai VLM MLLM 34 54 0 13 May 2023
Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphs Roei Herzig Alon Mendelson Leonid Karlinsky Assaf Arbelle Rogerio Feris Trevor Darrell Amir Globerson VLM 38 31 0 10 May 2023
SATIN: A Multi-Task Metadataset for Classifying Satellite Imagery using Vision-Language Models Jonathan Roberts Kai Han Samuel Albanie VLM 40 12 0 23 Apr 2023
OmniLabel: A Challenging Benchmark for Language-Based Object Detection S. Schulter G. VijayKumarB. Yumin Suh Konstantinos M. Dafnis Zhixing Zhang Shiyu Zhao Dimitris N. Metaxas ObjD 35 12 0 22 Apr 2023
Visual Instruction Tuning Haotian Liu Chunyuan Li Qingyang Wu Yong Jae Lee SyDa VLM MLLM 133 4,310 0 17 Apr 2023
V3Det: Vast Vocabulary Visual Detection Dataset Jiaqi Wang Pan Zhang Tao Chu Yuhang Cao Yujie Zhou Tong Wu Bin Wang Conghui He Dahua Lin VLM ObjD 29 52 0 07 Apr 2023
Vision-Language Models for Vision Tasks: A Survey Jingyi Zhang Jiaxing Huang Sheng Jin Shijian Lu VLM 43 486 0 03 Apr 2023
DIME-FM: DIstilling Multimodal and Efficient Foundation Models Ximeng Sun Pengchuan Zhang Peizhao Zhang Hardik Shah Kate Saenko Xide Xia VLM 25 20 0 31 Mar 2023
Going Beyond Nouns With Vision & Language Models Using Synthetic Data Paola Cascante-Bonilla Khaled Shehada James Smith Sivan Doveh Donghyun Kim ... Gül Varol A. Oliva Vicente Ordonez Rogerio Feris Leonid Karlinsky VLM SyDa 29 40 0 30 Mar 2023
Prompt Tuning based Adapter for Vision-Language Model Adaption Jingchen Sun Jiayu Qin Zihao Lin Changyou Chen VPVLM MLLM VLM 36 5 0 24 Mar 2023
A Large-scale Study of Spatiotemporal Representation Learning with a New Benchmark on Action Recognition Andong Deng Taojiannan Yang Chong Chen AI4TS 27 13 0 23 Mar 2023
A Simple Framework for Open-Vocabulary Segmentation and Detection Hao Zhang Feng Li Xueyan Zou Siyi Liu Chun-yue Li Jianfeng Gao Jianwei Yang Lei Zhang ObjD VLM 22 150 0 14 Mar 2023
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection Shilong Liu Zhaoyang Zeng Tianhe Ren Feng Li Hao Zhang ... Chun-yue Li Jianwei Yang Hang Su Jun Zhu Lei Zhang ObjD 106 1,823 0 09 Mar 2023
Learning Customized Visual Models with Retrieval-Augmented Knowledge Haotian Liu Kilho Son Jianwei Yang Ce Liu Jianfeng Gao Yong Jae Lee Chunyuan Li VLM 40 53 0 17 Jan 2023
GLIGEN: Open-Set Grounded Text-to-Image Generation Yuheng Li Haotian Liu Qingyang Wu Fangzhou Mu Jianwei Yang Jianfeng Gao Chunyuan Li Yong Jae Lee VLM 80 569 1 17 Jan 2023
Generalized Decoding for Pixel, Image, and Language Xueyan Zou Zi-Yi Dou Jianwei Yang Zhe Gan Linjie Li ... Lu Yuan Nanyun Peng Lijuan Wang Yong Jae Lee Jianfeng Gao VLM MLLM ObjD 21 241 0 21 Dec 2022
FlexiViT: One Model for All Patch Sizes Lucas Beyer Pavel Izmailov Alexander Kolesnikov Mathilde Caron Simon Kornblith Xiaohua Zhai Matthias Minderer Michael Tschannen Ibrahim M. Alabdulmohsin Filip Pavetić VLM 45 90 0 15 Dec 2022
Exploiting Category Names for Few-Shot Classification with Vision-Language Models Taihong Xiao Zirui Wang Liangliang Cao Jiahui Yu Shengyang Dai Ming Yang VLM MLLM 36 5 0 29 Nov 2022
Context-Aware Robust Fine-Tuning Xiaofeng Mao YueFeng Chen Xiaojun Jia Rong Zhang Hui Xue Zhao Li VLM CLIP 35 25 0 29 Nov 2022
ComCLIP: Training-Free Compositional Image and Text Matching Kenan Jiang Xuehai He Ruize Xu Qing Guo VLM CLIP CoGe 19 20 0 25 Nov 2022
Teaching Structured Vision&Language Concepts to Vision&Language Models Sivan Doveh Assaf Arbelle Sivan Harary Yikang Shen Roei Herzig ... Donghyun Kim Raja Giryes Rogerio Feris S. Ullman Leonid Karlinsky VLM CoGe 56 70 0 21 Nov 2022
Multitask Vision-Language Prompt Tuning Sheng Shen Shijia Yang Tianjun Zhang Bohan Zhai Joseph E. Gonzalez Kurt Keutzer Trevor Darrell VLM VPVLM 19 49 0 21 Nov 2022
Task Residual for Tuning Vision-Language Models Tao Yu Zhihe Lu Xin Jin Zhibo Chen Xinchao Wang VLM CLIP 24 82 0 18 Nov 2022
AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities Zhongzhi Chen Guangyi Liu Bo-Wen Zhang Fulong Ye Qinghong Yang Ledell Yu Wu VLM 37 80 0 12 Nov 2022
On the Feasibility of Cross-Task Transfer with Model-Based Reinforcement Learning Yifan Xu Nicklas Hansen Zirui Wang Yung-Chieh Chan H. Su Zhuowen Tu OffRL 31 15 0 19 Oct 2022
OmDet: Large-scale vision-language multi-dataset pre-training with multimodal detection network Tiancheng Zhao Peng Liu Kyusong Lee VLM MLLM ObjD 27 7 0 10 Sep 2022
SupMAE: Supervised Masked Autoencoders Are Efficient Vision Learners Feng Liang Yangguang Li Diana Marculescu SSL TPM ViT 51 22 0 28 May 2022
K-LITE: Learning Transferable Visual Models with External Knowledge Sheng Shen Chunyuan Li Xiaowei Hu Jianwei Yang Yujia Xie ... Ce Liu Kurt Keutzer Trevor Darrell Anna Rohrbach Jianfeng Gao CLIP VLM 36 83 0 20 Apr 2022