Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks

17 June 2022

Papers citing "Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks"

50 / 327 papers shown

Title
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond Jinze Bai Shuai Bai Shusheng Yang Shijie Wang Sinan Tan Peng Wang Junyang Lin Chang Zhou Jingren Zhou MLLM VLM ObjD 50 808 0 24 Aug 2023
DiffDis: Empowering Generative Diffusion Model with Cross-Modal Discrimination Capability Runhu Huang Jianhua Han Guansong Lu Xiaodan Liang Yihan Zeng Wei Zhang Hang Xu DiffM 23 2 0 18 Aug 2023
FocusFlow: Boosting Key-Points Optical Flow Estimation for Autonomous Driving Zhonghua Yi Haowen Shi Kailun Yang Zhijie Xu Yaozu Ye Ze Wang Huajian Ni Kaiwei Wang 3DPC 22 9 0 14 Aug 2023
Learning to Model the World with Language Jessy Lin Yuqing Du Olivia Watkins Danijar Hafner Pieter Abbeel Dan Klein Anca Dragan LM&Ro SyDa 38 51 0 31 Jul 2023
UnIVAL: Unified Model for Image, Video, Audio and Language Tasks Mustafa Shukor Corentin Dancette Alexandre Ramé Matthieu Cord MoMe MLLM 61 42 0 30 Jul 2023
Towards Generalist Biomedical AI Tao Tu Shekoofeh Azizi Danny Driess M. Schaekermann Mohamed Amin ... Yossi Matias K. Singhal Peter R. Florence Alan Karthikesalingam Vivek Natarajan LM&MA MedIm AI4MH 49 243 0 26 Jul 2023
Described Object Detection: Liberating Object Detection with Flexible Expressions Chi Xie Zhao Zhang YiXuan Wu Feng Zhu Rui Zhao Shuang Liang ObjD 39 31 0 24 Jul 2023
UniFormaly: Towards Task-Agnostic Unified Framework for Visual Anomaly Detection Yujin Lee Harin Lim Seoyoon Jang H. Yoon 32 4 0 24 Jul 2023
Multimodal LLMs for health grounded in individual-specific data Anastasiya Belyaeva J. Cosentino F. Hormozdiari Krish Eswaran S. Shetty Greg C. Corrado Andrew Carroll Cory Y. McLean N. Furlotte LM&MA 33 49 0 18 Jul 2023
PAT: Parallel Attention Transformer for Visual Question Answering in Vietnamese Nghia Hieu Nguyen Kiet Van Nguyen 16 2 0 17 Jul 2023
Leveraging Vision-Language Foundation Models for Fine-Grained Downstream Tasks Denis Coquenet Clément Rambour Emanuele Dalsasso Nicolas Thome MLLM CLIP VLM 37 1 0 13 Jul 2023
Objaverse-XL: A Universe of 10M+ 3D Objects Matt Deitke Ruoshi Liu Matthew Wallingford Huong Ngo Oscar Michel ... Carl Vondrick Georgia Gkioxari Kiana Ehsani Ludwig Schmidt Ali Farhadi 25 381 0 11 Jul 2023
Emu: Generative Pretraining in Multimodality Quan-Sen Sun Qiying Yu Yufeng Cui Fan Zhang Xiaosong Zhang Yueze Wang Hongcheng Gao Jingjing Liu Tiejun Huang Xinlong Wang MLLM 40 126 0 11 Jul 2023
Building Cooperative Embodied Agents Modularly with Large Language Models Hongxin Zhang Weihua Du Jiaming Shan Qinhong Zhou Yilun Du J. Tenenbaum Tianmin Shu Chuang Gan LLMAG LM&Ro 59 157 0 05 Jul 2023
Visual Instruction Tuning with Polite Flamingo Delong Chen Jianfeng Liu Wenliang Dai Baoyuan Wang MLLM 34 42 0 03 Jul 2023
JourneyDB: A Benchmark for Generative Image Understanding Keqiang Sun Junting Pan Yuying Ge Hao Li Haodong Duan ... Yi Wang Jifeng Dai Yu Qiao Limin Wang Hongsheng Li 54 103 0 03 Jul 2023
An Efficient General-Purpose Modular Vision Model via Multi-Task Heterogeneous Training Z. Chen Mingyu Ding Songlin Yang Wei Zhan Masayoshi Tomizuka Erik Learned-Miller Chuang Gan MoE 26 8 0 29 Jun 2023
Unified Language Representation for Question Answering over Text, Tables, and Images Yu Bowen Cheng Fu Haiyang Yu Fei Huang Yongbin Li LMTD 30 20 0 29 Jun 2023
Semi-supervised Multimodal Representation Learning through a Global Workspace Benjamin Devillers Léopold Maytié R. V. Rullen SSL 27 5 0 27 Jun 2023
COSA: Concatenated Sample Pretrained Vision-Language Foundation Model Sihan Chen Xingjian He Handong Li Xiaojie Jin Jiashi Feng Jiaheng Liu VLM CLIP 30 8 0 15 Jun 2023
Towards AGI in Computer Vision: Lessons Learned from GPT and Large Language Models Lingxi Xie Longhui Wei Xiaopeng Zhang Kaifeng Bi Xiaotao Gu Jianlong Chang Qi Tian 35 7 0 14 Jun 2023
AVIS: Autonomous Visual Information Seeking with Large Language Model Agent Ziniu Hu Ahmet Iscen Chen Sun Kai-Wei Chang Yizhou Sun David A. Ross Cordelia Schmid Alireza Fathi 31 11 0 13 Jun 2023
Global and Local Semantic Completion Learning for Vision-Language Pre-training Rong-Cheng Tu Yatai Ji Jie Jiang Weijie Kong Chengfei Cai Wenzhe Zhao Hongfa Wang Yujiu Yang Wei Liu VLM 24 2 0 12 Jun 2023
Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards Alexandre Ramé Guillaume Couairon Mustafa Shukor Corentin Dancette Jean-Baptiste Gaya Laure Soulier Matthieu Cord MoMe 35 136 0 07 Jun 2023
Q: How to Specialize Large Vision-Language Models to Data-Scarce VQA Tasks? A: Self-Train on Unlabeled Images! Zaid Khan B. Vijaykumar S. Schulter Xiang Yu Y. Fu Manmohan Chandraker VLM MLLM 32 17 0 06 Jun 2023
Unifying (Machine) Vision via Counterfactual World Modeling Daniel M. Bear Kevin T. Feigelis Honglin Chen Wanhee Lee R. Venkatesh Klemen Kotar Alex Durango Daniel L. K. Yamins VGen 25 13 0 02 Jun 2023
Bytes Are All You Need: Transformers Operating Directly On File Bytes Maxwell Horton Sachin Mehta Ali Farhadi Mohammad Rastegari VLM 22 6 0 31 May 2023
There is more to graphs than meets the eye: Learning universal features with self-supervision L. Das Sai Munikoti M. Halappanavar SSL OOD 30 1 0 31 May 2023
Generate then Select: Open-ended Visual Question Answering Guided by World Knowledge Xingyu Fu Shenmin Zhang Gukyeong Kwon Pramuditha Perera Henghui Zhu ... Zhiguo Wang Vittorio Castelli Patrick K. L. Ng Dan Roth Bing Xiang 29 19 0 30 May 2023
GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction Rui Yang Lin Song Yanwei Li Sijie Zhao Yixiao Ge Xiu Li Ying Shan SyDa MLLM 36 209 0 30 May 2023
PaLI-X: On Scaling up a Multilingual Vision and Language Model Xi Chen Josip Djolonga Piotr Padlewski Basil Mustafa Soravit Changpinyo ... Mojtaba Seyedhosseini A. Angelova Xiaohua Zhai N. Houlsby Radu Soricut VLM 68 189 0 29 May 2023
Deeply Coupled Cross-Modal Prompt Learning Xuejing Liu Wei Tang Jinghui Lu Rui Zhao Zhaojun Guo Fei Tan VLM 25 17 0 29 May 2023
Generating Images with Multimodal Language Models Jing Yu Koh Daniel Fried Ruslan Salakhutdinov MLLM 28 241 0 26 May 2023
BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks Kai Zhang Jun Yu Eashan Adhikarla Rong-Er Zhou Zhilin Yan ... Xun Chen Yong Chen Quanzheng Li Hongfang Liu Lichao Sun LM&MA MedIm 37 157 0 26 May 2023
LANISTR: Multimodal Learning from Structured and Unstructured Data Sayna Ebrahimi Sercan Ö. Arik Yihe Dong Tomas Pfister 20 4 0 26 May 2023
Exploring Diverse In-Context Configurations for Image Captioning Xu Yang Yongliang Wu Mingzhuo Yang Haokun Chen Xin Geng MLLM 27 51 0 24 May 2023
Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining Emanuele Bugliarello Aida Nematzadeh Lisa Anne Hendricks SSL 30 5 0 23 May 2023
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data Ziyi Yang Mahmoud Khademi Yichong Xu Reid Pryzant Yuwei Fang ... Yu Shi Lu Yuan Takuya Yoshioka Michael Zeng Xuedong Huang 17 2 0 21 May 2023
Multimodal Web Navigation with Instruction-Finetuned Foundation Models Hiroki Furuta Kuang-Huei Lee Ofir Nachum Yutaka Matsuo Aleksandra Faust S. Gu Izzeddin Gur LM&Ro 36 93 0 19 May 2023
TreePrompt: Learning to Compose Tree Prompts for Explainable Visual Grounding Chenchi Zhang Jun Xiao Lei Chen Jian Shao Long Chen VLM LRM 32 2 0 19 May 2023
LLM-CXR: Instruction-Finetuned LLM for CXR Image Understanding and Generation Suhyeon Lee Won Jun Kim Jinho Chang Jong Chul Ye MedIm 32 48 0 19 May 2023
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks Wen Wang Zhe Chen Xiaokang Chen Jiannan Wu Xizhou Zhu ... Ping Luo Tong Lu Jie Zhou Yu Qiao Jifeng Dai MLLM VLM 35 460 0 18 May 2023
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities Peng Wang Shijie Wang Junyang Lin Shuai Bai Xiaohuan Zhou Jingren Zhou Xinggang Wang Chang Zhou VLM MLLM ObjD 48 115 0 18 May 2023
Segment Any Anomaly without Training via Hybrid Prompt Regularization Yunkang Cao Xiaohao Xu Chen Sun Y. Cheng Zongwei Du Liang Gao Nong Sang VLM 37 70 0 18 May 2023
Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Prompts Zhaoyang Zhang Yantao Shen Kunyu Shi Zhaowei Cai Jun Fang Siqi Deng Hao Yang Davide Modolo Z. Tu Stefano Soatto VLM 28 2 0 11 May 2023
Self-Chained Image-Language Model for Video Localization and Question Answering Shoubin Yu Jaemin Cho Prateek Yadav Joey Tianyi Zhou 54 130 0 11 May 2023
OpenViVQA: Task, Dataset, and Multimodal Fusion Models for Visual Question Answering in Vietnamese Nghia Hieu Nguyen Duong T.D. Vo Kiet Van Nguyen Ngan Luu-Thuy Nguyen 29 18 0 07 May 2023
An Empirical Study of Multimodal Model Merging Yi-Lin Sung Linjie Li Kevin Qinghong Lin Zhe Gan Joey Tianyi Zhou Lijuan Wang MoMe 20 40 0 28 Apr 2023
$π$ -Tuning: Transferring Multimodal Foundation Models with Optimal Multi-task Interpolation Chengyue Wu Teng Wang Yixiao Ge Zeyu Lu Rui-Zhi Zhou Ying Shan Ping Luo MoMe 88 35 0 27 Apr 2023
Transformer-Based Visual Segmentation: A Survey Xiangtai Li Henghui Ding Haobo Yuan Wenwei Zhang Jiangmiao Pang Guangliang Cheng Kai-xiang Chen Ziwei Liu Chen Change Loy ViT MedIm 42 132 0 19 Apr 2023