UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling

23 November 2021

Zicheng Liu

Papers citing "UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling"

50 / 104 papers shown

Title
Enhancing Visual Grounding and Generalization: A Multi-Task Cycle Training Approach for Vision-Language Models Xiaoyu Yang Lijian Xu Hao Sun Hongsheng Li Shaoting Zhang ObjD 25 4 0 21 Nov 2023
DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback Yangyi Chen Karan Sikka Michael Cogswell Heng Ji Ajay Divakaran 29 58 0 16 Nov 2023
GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation An Yan Zhengyuan Yang Wanrong Zhu K. Lin Linjie Li ... Yiwu Zhong Julian McAuley Jianfeng Gao Zicheng Liu Lijuan Wang LLMAG LM&Ro 33 100 0 13 Nov 2023
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks Bin Xiao Haiping Wu Weijian Xu Xiyang Dai Houdong Hu Yumao Lu Michael Zeng Ce Liu Lu Yuan VLM 36 143 0 10 Nov 2023
NExT-Chat: An LMM for Chat, Detection and Segmentation Ao Zhang Yuan Yao Wei Ji Zhiyuan Liu Tat-Seng Chua MLLM VLM 48 51 0 08 Nov 2023
Multitask Multimodal Prompted Training for Interactive Embodied Task Completion Georgios Pantazopoulos Malvina Nikandrou Amit Parekh Bhathiya Hemanthage Arash Eshghi Ioannis Konstas Verena Rieser Oliver Lemon Alessandro Suglia LM&Ro 31 7 0 07 Nov 2023
CLIP meets Model Zoo Experts: Pseudo-Supervision for Visual Enhancement Mohammadreza Salehi Mehrdad Farajtabar Maxwell Horton Fartash Faghri Hadi Pouransari Raviteja Vemulapalli Oncel Tuzel Ali Farhadi Mohammad Rastegari Sachin Mehta CLIP VLM 48 1 0 21 Oct 2023
Ferret: Refer and Ground Anything Anywhere at Any Granularity Haoxuan You Haotian Zhang Zhe Gan Xianzhi Du Bowen Zhang Zirui Wang Liangliang Cao Shih-Fu Chang Yinfei Yang ObjD MLLM VLM 22 301 0 11 Oct 2023
Reinforced UI Instruction Grounding: Towards a Generic UI Task Automation API Zhizheng Zhang Wenxuan Xie Xiaoyi Zhang Yan Lu 26 10 0 07 Oct 2023
VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Models Ziyi Yin Muchao Ye Tianrong Zhang Tianyu Du Jinguo Zhu Han Liu Jinghui Chen Ting Wang Fenglong Ma AAML VLM CoGe 33 36 0 07 Oct 2023
Expedited Training of Visual Conditioned Language Generation via Redundancy Reduction Yiren Jian Tingkai Liu Yunzhe Tao Chunhui Zhang Soroush Vosoughi HX Yang VLM 17 7 0 05 Oct 2023
SCOB: Universal Text Understanding via Character-wise Supervised Contrastive Learning with Online Text Rendering for Bridging Domain Gap Daehee Kim Yoon Kim Donghyun Kim Yumin Lim Geewook Kim Taeho Kil 28 3 0 21 Sep 2023
Measuring and Improving Chain-of-Thought Reasoning in Vision-Language Models Yangyi Chen Karan Sikka Michael Cogswell Heng Ji Ajay Divakaran LRM 36 24 0 08 Sep 2023
A Joint Study of Phrase Grounding and Task Performance in Vision and Language Models Noriyuki Kojima Hadar Averbuch-Elor Yoav Artzi 26 2 0 06 Sep 2023
Towards Unified Token Learning for Vision-Language Tracking Yaozong Zheng Bineng Zhong Qihua Liang Guorong Li Rongrong Ji Xianxian Li 27 28 0 27 Aug 2023
SCoRD: Subject-Conditional Relation Detection with Text-Augmented Data Ziyan Yang Kushal Kafle Zhe-nan Lin Scott D. Cohen Zhihong Ding Vicente Ordonez 23 1 0 24 Aug 2023
MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities Weihao Yu Zhengyuan Yang Linjie Li Jianfeng Wang Kevin Qinghong Lin Zicheng Liu Xinchao Wang Lijuan Wang MLLM 43 607 0 04 Aug 2023
Advancing Visual Grounding with Scene Knowledge: Benchmark and Method Zhihong Chen Ruifei Zhang Yibing Song Xiang Wan Guanbin Li 22 15 0 21 Jul 2023
ChatSpot: Bootstrapping Multimodal LLMs via Precise Referring Instruction Tuning Liang Zhao En Yu Zheng Ge Jinrong Yang Hao-Ran Wei ... Jian‐Yuan Sun Yuang Peng Runpei Dong Chunrui Han Xiangyu Zhang MLLM LRM 36 53 0 18 Jul 2023
Bootstrapping Vision-Language Learning with Decoupled Language Pre-training Yiren Jian Chongyang Gao Soroush Vosoughi VLM MLLM 32 25 0 13 Jul 2023
EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone Shraman Pramanick Yale Song Sayan Nag Kevin Qinghong Lin Hardik Shah Mike Zheng Shou Ramalingam Chellappa Pengchuan Zhang VLM 39 87 0 11 Jul 2023
Vision Language Transformers: A Survey Clayton Fields C. Kennington VLM 25 5 0 06 Jul 2023
Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic Ke Chen Zhao Zhang Weili Zeng Richong Zhang Feng Zhu Rui Zhao ObjD 42 596 0 27 Jun 2023
Kosmos-2: Grounding Multimodal Large Language Models to the World Zhiliang Peng Wenhui Wang Li Dong Y. Hao Shaohan Huang Shuming Ma Furu Wei MLLM ObjD VLM 41 695 0 26 Jun 2023
Responsible Task Automation: Empowering Large Language Models as Responsible Task Automators Zhizheng Zhang Xiaoyi Zhang Wenxuan Xie Yan Lu 21 13 0 02 Jun 2023
Deeply Coupled Cross-Modal Prompt Learning Xuejing Liu Wei Tang Jinghui Lu Rui Zhao Zhaojun Guo Fei Tan VLM 20 17 0 29 May 2023
Visual Programming for Text-to-Image Generation and Evaluation Jaemin Cho Abhaysinh Zala Joey Tianyi Zhou MLLM 21 50 0 24 May 2023
VIP5: Towards Multimodal Foundation Models for Recommendation Shijie Geng Juntao Tan Shuchang Liu Zuohui Fu Yongfeng Zhang 26 69 0 23 May 2023
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data Ziyi Yang Mahmoud Khademi Yichong Xu Reid Pryzant Yuwei Fang ... Yu Shi Lu Yuan Takuya Yoshioka Michael Zeng Xuedong Huang 17 2 0 21 May 2023
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks Wen Wang Zhe Chen Xiaokang Chen Jiannan Wu Xizhou Zhu ... Ping Luo Tong Lu Jie Zhou Yu Qiao Jifeng Dai MLLM VLM 35 457 0 18 May 2023
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities Peng Wang Shijie Wang Junyang Lin Shuai Bai Xiaohuan Zhou Jingren Zhou Xinggang Wang Chang Zhou VLM MLLM ObjD 42 114 0 18 May 2023
Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Prompts Zhaoyang Zhang Yantao Shen Kunyu Shi Zhaowei Cai Jun Fang Siqi Deng Hao-Yu Yang Davide Modolo Z. Tu Stefano Soatto VLM 28 2 0 11 May 2023
VideoOFA: Two-Stage Pre-Training for Video-to-Text Generation Xilun Chen L. Yu Wenhan Xiong Barlas Ouguz Yashar Mehdad Wen-tau Yih VGen 26 3 0 04 May 2023
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Sihan Chen Xingjian He Longteng Guo Xinxin Zhu Weining Wang Jinhui Tang Jinhui Tang VLM 31 102 0 17 Apr 2023
Towards Unified Scene Text Spotting based on Sequence Generation Taeho Kil Seonghyeon Kim Sukmin Seo Yoon Kim Daehee Kim 72 20 0 07 Apr 2023
What, when, and where? -- Self-Supervised Spatio-Temporal Grounding in Untrimmed Multi-Action Videos from Narrated Instructions Brian Chen Nina Shvetsova Andrew Rouditchenko D. Kondermann Samuel Thomas Shih-Fu Chang Rogerio Feris James R. Glass Hilde Kuehne 37 7 0 29 Mar 2023
PolyFormer: Referring Image Segmentation as Sequential Polygon Generation Jiang Liu Hui Ding Zhaowei Cai Yuting Zhang R. Satzoda Vijay Mahadevan R. Manmatha ObjD 20 120 0 14 Feb 2023
Generalized Decoding for Pixel, Image, and Language Xueyan Zou Zi-Yi Dou Jianwei Yang Zhe Gan Linjie Li ... Lu Yuan Nanyun Peng Lijuan Wang Yong Jae Lee Jianfeng Gao VLM MLLM ObjD 21 240 0 21 Dec 2022
ReCo: Region-Controlled Text-to-Image Generation Zhengyuan Yang Jianfeng Wang Zhe Gan Linjie Li Kevin Qinghong Lin ... Nan Duan Zicheng Liu Ce Liu Michael Zeng Lijuan Wang DiffM 56 140 0 23 Nov 2022
Uni-Perceiver v2: A Generalist Model for Large-Scale Vision and Vision-Language Tasks Hao Li Jinguo Zhu Xiaohu Jiang Xizhou Zhu Hongsheng Li ... Xiaohua Wang Yu Qiao Xiaogang Wang Wenhai Wang Jifeng Dai MLLM 18 55 0 17 Nov 2022
PromptCap: Prompt-Guided Task-Aware Image Captioning Yushi Hu Hang Hua Zhengyuan Yang Weijia Shi Noah A. Smith Jiebo Luo 45 101 0 15 Nov 2022
VoLTA: Vision-Language Transformer with Weakly-Supervised Local-Feature Alignment Shraman Pramanick Li Jing Sayan Nag Jiachen Zhu Hardik Shah Yann LeCun Ramalingam Chellappa 26 21 0 09 Oct 2022
Dynamic MDETR: A Dynamic Multimodal Transformer Decoder for Visual Grounding Fengyuan Shi Ruopeng Gao Weilin Huang Limin Wang 27 23 0 28 Sep 2022
Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks Jiasen Lu Christopher Clark Rowan Zellers Roozbeh Mottaghi Aniruddha Kembhavi ObjD VLM MLLM 51 392 0 17 Jun 2022
LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling Linjie Li Zhe Gan Kevin Qinghong Lin Chung-Ching Lin Zicheng Liu Ce Liu Lijuan Wang MLLM VLM 20 81 0 14 Jun 2022
PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models Yuan Yao Qi-An Chen Ao Zhang Wei Ji Zhiyuan Liu Tat-Seng Chua Maosong Sun VLM MLLM 26 38 0 23 May 2022
Training Vision-Language Transformers from Captions Liangke Gui Yingshan Chang Qiuyuan Huang Subhojit Som Alexander G. Hauptmann Jianfeng Gao Yonatan Bisk VLM ViT 174 11 0 19 May 2022
A Survivor in the Era of Large-Scale Pretraining: An Empirical Study of One-Stage Referring Expression Comprehension Gen Luo Yiyi Zhou Jiamu Sun Xiaoshuai Sun Rongrong Ji ObjD 21 10 0 17 Apr 2022
I-Tuning: Tuning Frozen Language Models with Image for Lightweight Image Captioning Ziyang Luo Zhipeng Hu Yadong Xi Rongsheng Zhang Jing Ma VLM 17 13 0 14 Feb 2022
Pix2seq: A Language Modeling Framework for Object Detection Ting-Li Chen Saurabh Saxena Lala Li David J. Fleet Geoffrey E. Hinton MLLM ViT VLM 241 344 0 22 Sep 2021