v1v2 (latest)

MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding

26 April 2021

ArXiv (abs)PDF HTML Github (1008★)

Papers citing "MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding"

50 / 616 papers shown

Title
Contextual Object Detection with Multimodal Large Language Models Yuhang Zang Wei Li Jun Han Kaiyang Zhou Chen Change Loy ObjD VLM MLLM 118 88 0 29 May 2023
Z-GMOT: Zero-shot Generic Multiple Object Tracking Kim Hoang Tran Anh Duy Le Dinh Tien-Phat Nguyen Thinh Phan Pha Nguyen Khoa Luu Don Adjeroh Gianfranco Doretto Ngan Hoang Le VOT 94 7 0 28 May 2023
Modularized Zero-shot VQA with Pre-trained Models Rui Cao Jing Jiang LRM 89 3 0 27 May 2023
Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation Shilin Yan Renrui Zhang Ziyu Guo Wenchao Chen Wei Zhang Hongyang Li Yu Qiao Hao Dong Zhongjiang He Peng Gao VOS 117 37 0 25 May 2023
Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentation Chang Liu Henghui Ding Yulun Zhang Xudong Jiang 109 50 0 24 May 2023
GRILL: Grounded Vision-language Pre-training via Aligning Text and Image Regions Woojeong Jin Subhabrata Mukherjee Yu Cheng Yelong Shen Weizhu Chen Ahmed Hassan Awadallah Damien Jose Xiang Ren ObjD VLM 121 8 0 24 May 2023
Cross3DVG: Cross-Dataset 3D Visual Grounding on Different RGB-D Scans Taiki Miyanishi Daich Azuma Shuhei Kurita M. Kawanabe 77 2 0 23 May 2023
Perception Test: A Diagnostic Benchmark for Multimodal Video Models Viorica Puatruaucean Lucas Smaira Ankush Gupta Adrià Recasens Continente L. Markeeva ... Y. Aytar Simon Osindero Dima Damen Andrew Zisserman João Carreira VLM 235 179 0 23 May 2023
Type-to-Track: Retrieve Any Object via Prompt-based Tracking Pha Nguyen Kha Gia Quach Kris Kitani Khoa Luu 97 20 0 22 May 2023
Multimodal Web Navigation with Instruction-Finetuned Foundation Models Hiroki Furuta Kuang-Huei Lee Ofir Nachum Yutaka Matsuo Aleksandra Faust S. Gu Izzeddin Gur LM&Ro 179 103 0 19 May 2023
TreePrompt: Learning to Compose Tree Prompts for Explainable Visual Grounding Chenchi Zhang Jun Xiao Lei Chen Jian Shao Long Chen VLM LRM 85 2 0 19 May 2023
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks Wen Wang Zhe Chen Xiaokang Chen Jiannan Wu Xizhou Zhu ... Ping Luo Tong Lu Jie Zhou Yu Qiao Jifeng Dai MLLM VLM 118 494 0 18 May 2023
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities Peng Wang Shijie Wang Junyang Lin Shuai Bai Xiaohuan Zhou Jingren Zhou Xinggang Wang Chang Zhou VLM MLLM ObjD 151 122 0 18 May 2023
Visual Question Answering: A Survey on Techniques and Common Trends in Recent Literature Ana Claudia Akemi Matsuki de Faria Felype de Castro Bastos Jose Victor Nogueira Alves da Silva Vitor Lopes Fabris Valeska Uchôa Décio Gonccalves de Aguiar Neto C. F. G. Santos 68 27 0 18 May 2023
Annotation-free Audio-Visual Segmentation Jinxian Liu Yu Wang Chen Ju Chaofan Ma Ya Zhang Weidi Xie VOS VLM 109 30 0 18 May 2023
Weakly-Supervised Visual-Textual Grounding with Semantic Prior Refinement Davide Rigoni Luca Parolari Luciano Serafini A. Sperduti Lamberto Ballan 85 1 0 18 May 2023
UniS-MMC: Multimodal Classification via Unimodality-supervised Multimodal Contrastive Learning Heqing Zou Meng Shen Chen Chen Yuchen Hu D. Rajan Chng Eng Siong SSL 100 17 0 16 May 2023
CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Grounding Linhui Xiao Xiaoshan Yang Fang Peng Ming Yan Yaowei Wang Changsheng Xu ObjD VLM 119 31 0 15 May 2023
COLA: A Benchmark for Compositional Text-to-image Retrieval Arijit Ray Filip Radenovic Abhimanyu Dubey Bryan A. Plummer Ranjay Krishna Kate Saenko CoGe VLM 112 38 0 05 May 2023
Unified Model Learning for Various Neural Machine Translation Yunlong Liang Fandong Meng Jinan Xu Jiaan Wang Jinan Xu Jie Zhou 83 1 0 04 May 2023
Energy-based Models are Zero-Shot Planners for Compositional Scene Rearrangement N. Gkanatsios Ayush Jain Zhou Xian Yunchu Zhang C. Atkeson Katerina Fragkiadaki LM&Ro 160 33 0 27 Apr 2023
$π$ -Tuning: Transferring Multimodal Foundation Models with Optimal Multi-task Interpolation Chengyue Wu Teng Wang Yixiao Ge Zeyu Lu Rui-Zhi Zhou Ying Shan Ping Luo MoMe 145 37 0 27 Apr 2023
Zero-shot Unsupervised Transfer Instance Segmentation Gyungin Shin Samuel Albanie Weidi Xie ISeg VLM 145 5 0 27 Apr 2023
Multimodal Grounding for Embodied AI via Augmented Reality Headsets for Natural Language Driven Task Planning Selma Wanna Fabian Parra R. Valner Karl Kruusamäe Mitch Pryor LM&Ro 68 2 0 26 Apr 2023
A Cookbook of Self-Supervised Learning Randall Balestriero Mark Ibrahim Vlad Sobal Ari S. Morcos Shashank Shekhar ... Pierre Fernandez Amir Bar Hamed Pirsiavash Yann LeCun Micah Goldblum SyDa FedML SSL 161 285 0 24 Apr 2023
OmniLabel: A Challenging Benchmark for Language-Based Object Detection S. Schulter G. VijayKumarB. Yumin Suh Konstantinos M. Dafnis Zhixing Zhang Shiyu Zhao Dimitris N. Metaxas ObjD 75 12 0 22 Apr 2023
Transformer-Based Visual Segmentation: A Survey Xiangtai Li Henghui Ding Haobo Yuan Wenwei Zhang Jiangmiao Pang Guangliang Cheng Kai-xiang Chen Ziwei Liu Chen Change Loy ViT MedIm 170 147 0 19 Apr 2023
Delving into Shape-aware Zero-shot Semantic Segmentation Xinyu Liu Beiwen Tian Zhen Wang Rui Wang Kehua Sheng Bo Zhang Hao Zhao Guyue Zhou VLM 106 20 0 17 Apr 2023
On the Opportunities and Challenges of Foundation Models for Geospatial Artificial Intelligence Gengchen Mai Weiming Huang Jin Sun Suhang Song Deepak Mishra ... Yingjie Hu Chris Cundy Ziyuan Li Rui Zhu Ni Lao AI4CE 122 134 0 13 Apr 2023
What does CLIP know about a red circle? Visual prompt engineering for VLMs Aleksandar Shtedritski Christian Rupprecht Andrea Vedaldi VLM MLLM 111 162 0 13 Apr 2023
Verbs in Action: Improving verb understanding in video-language models Liliane Momeni Mathilde Caron Arsha Nagrani Andrew Zisserman Cordelia Schmid 105 71 0 13 Apr 2023
WildRefer: 3D Object Localization in Large-scale Dynamic Scenes with Multi-modal Visual Data and Natural Language Zhe Lin Xidong Peng Peishan Cong Ge Zheng Yujin Sun Yuenan Hou Xinge Zhu Sibei Yang Yuexin Ma VGen 135 5 0 12 Apr 2023
MoMo: A shared encoder Model for text, image and multi-Modal representations Rakesh Chada Zhao-Heng Zheng P. Natarajan ViT 64 4 0 11 Apr 2023
Detection Transformer with Stable Matching Siyi Liu Tianhe Ren Jia-Yu Chen Zhaoyang Zeng Hao Zhang ... Hongyang Li Jun Huang Hang Su Jun Zhu Lei Zhang 80 36 0 10 Apr 2023
DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-training via Word-Region Alignment Lewei Yao Jianhua Han Xiaodan Liang Danqian Xu Wei Zhang Zhenguo Li Hang Xu VLM ObjD CLIP 128 79 0 10 Apr 2023
CAVL: Learning Contrastive and Adaptive Representations of Vision and Language Shentong Mo Jingfei Xia Ihor Markevych CLIP VLM 62 1 0 10 Apr 2023
ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenes Ran Gong Jiangyong Huang Yizhou Zhao Haoran Geng Xiaofeng Gao ... Ziheng Zhou D. Terzopoulos Song-Chun Zhu Baoxiong Jia Siyuan Huang LM&Ro 108 50 0 09 Apr 2023
Mitigating Spurious Correlations in Multi-modal Models during Fine-tuning Yu Yang Besmira Nushi Hamid Palangi Baharan Mirzasoleiman 103 39 0 08 Apr 2023
V3Det: Vast Vocabulary Visual Detection Dataset Jiaqi Wang Pan Zhang Tao Chu Yuhang Cao Yujie Zhou Tong Wu Bin Wang Conghui He Dahua Lin VLM ObjD 119 55 0 07 Apr 2023
Learning Instance-Level Representation for Large-Scale Multi-Modal Pretraining in E-commerce Yang Jin Yongzhi Li Zehuan Yuan Yadong Mu 73 14 0 06 Apr 2023
Learning to Name Classes for Vision and Language Models Sarah Parisot Yongxin Yang Jingyu Sun VLM 89 10 0 04 Apr 2023
Locate Then Generate: Bridging Vision and Language with Bounding Box for Scene-Text VQA Yongxin Zhu Ziqiang Liu Yukang Liang Xin Li Hao Liu Changcun Bao Linli Xu 60 7 0 04 Apr 2023
Probabilistic Prompt Learning for Dense Prediction Hyeongjun Kwon Taeyong Song Somi Jeong Jin-Hwa Kim Jinhyun Jang Kwanghoon Sohn VLM 106 19 0 03 Apr 2023
Vision-Language Models for Vision Tasks: A Survey Jingyi Zhang Jiaxing Huang Sheng Jin Shijian Lu VLM 167 553 0 03 Apr 2023
What, when, and where? -- Self-Supervised Spatio-Temporal Grounding in Untrimmed Multi-Action Videos from Narrated Instructions Brian Chen Nina Shvetsova Andrew Rouditchenko D. Kondermann Samuel Thomas Shih-Fu Chang Rogerio Feris James R. Glass Hilde Kuehne 112 7 0 29 Mar 2023
ViewRefer: Grasp the Multi-view Knowledge for 3D Visual Grounding with GPT and Prototype Guidance Zoey Guo Yiwen Tang Renrui Zhang Dong Wang Zhigang Wang Bin Zhao Xuelong Li 117 60 0 29 Mar 2023
Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention Sounak Mondal Zhibo Yang Seoyoung Ahn Dimitris Samaras G. Zelinsky Minh Hoai 89 31 0 27 Mar 2023
Query-Dependent Video Representation for Moment Retrieval and Highlight Detection WonJun Moon Sangeek Hyun S. Park Dongchan Park Jae-Pil Heo ViT 107 115 0 24 Mar 2023
Open-Vocabulary Object Detection using Pseudo Caption Labels Han-Cheol Cho Won Young Jhoo Woohyun Kang Byungseok Roh VLM ObjD 67 20 0 23 Mar 2023
LD-ZNet: A Latent Diffusion Approach for Text-Based Image Segmentation K. Pnvr Bharat Singh P. Ghosh Behjat Siddiquie David Jacobs DiffM 86 29 0 22 Mar 2023