v1v2 (latest)

Fusion of Detected Objects in Text for Visual Question Answering

14 August 2019

ArXiv (abs)PDF HTML Github (1675★)

Papers citing "Fusion of Detected Objects in Text for Visual Question Answering"

50 / 108 papers shown

Title
Memory-Augmented Multimodal LLMs for Surgical VQA via Self-Contained Inquiry Wenjun Hou Yi Cheng Kaishuai Xu Yan Hu Wenjie Li Jiang-Dong Liu 65 1 0 17 Nov 2024
Vision Model Pre-training on Interleaved Image-Text Data via Latent Compression Learning Chenyu Yang Xizhou Zhu Jinguo Zhu Weijie Su Junjie Wang ... Lewei Lu Bin Li Jie Zhou Yu Qiao Jifeng Dai VLM CLIP 87 6 0 11 Jun 2024
EventLens: Leveraging Event-Aware Pretraining and Cross-modal Linking Enhances Visual Commonsense Reasoning Mingjie Ma Zhihuan Yu Yichao Ma Guohui Li LRM 73 1 0 22 Apr 2024
FSMR: A Feature Swapping Multi-modal Reasoning Approach with Joint Textual and Visual Clues Shuang Li Jiahua Wang Lijie Wen LRM 53 0 0 29 Mar 2024
Demonstrating and Reducing Shortcuts in Vision-Language Representation Learning Maurits J. R. Bleeker Mariya Hendriksen Andrew Yates Maarten de Rijke VLM 97 2 0 27 Feb 2024
$$\mathbb{VD}$-$\mathbb{GR}$: Boosting $\mathbb{V}$isual $\mathbb{D}$ialog with Cascaded Spatial-Temporal Multi-Modal $\mathbb{GR}$aphs$ $\mathbb{VD}$ - $\mathbb{GR}$ : Boosting $\mathbb{V}$ isual $\mathbb{D}$ ialog with Cascaded Spatial-Temporal Multi-Modal $\mathbb{GR}$ aphs Adnen Abdessaied Lei Shi Andreas Bulling 3DH 58 4 0 25 Oct 2023
UNK-VQA: A Dataset and a Probe into the Abstention Ability of Multi-modal Large Models Yanyang Guo Fangkai Jiao Zhiqi Shen Liqiang Nie Mohan S. Kankanhalli MLLM 87 7 0 17 Oct 2023
ELIP: Efficient Language-Image Pre-training with Fewer Vision Tokens Yangyang Guo Haoyu Zhang Yongkang Wong Liqiang Nie Mohan Kankanhalli VLM 69 3 0 28 Sep 2023
Separate and Locate: Rethink the Text in Text-based Visual Question Answering Chengyang Fang Jiangnan Li Liang Li Can Ma Dayong Hu 83 13 0 31 Aug 2023
MoviePuzzle: Visual Narrative Reasoning through Multimodal Order Learning Jianghui Wang Yuxuan Wang Dongyan Zhao Zilong Zheng 87 1 0 04 Jun 2023
Using Visual Cropping to Enhance Fine-Detail Question Answering of BLIP-Family Models Jiarui Zhang Mahyar Khayatkhoei P. Chhikara Filip Ilievski 56 1 0 31 May 2023
Deeply Coupled Cross-Modal Prompt Learning Xuejing Liu Wei Tang Jinghui Lu Rui Zhao Zhaojun Guo Fei Tan VLM 61 17 0 29 May 2023
ArK: Augmented Reality with Knowledge Interactive Emergent Ability Qiuyuan Huang Jinho Park Abhinav Gupta Paul N. Bennett Ran Gong ... Baolin Peng O. Mohammed C. Pal Yejin Choi Jianfeng Gao 119 6 0 01 May 2023
Enhancing object detection robustness: A synthetic and natural perturbation approach N. Premakumara B. Jalaeian N. Suri H. Samani 48 3 0 20 Apr 2023
Probabilistic Prompt Learning for Dense Prediction Hyeongjun Kwon Taeyong Song Somi Jeong Jin-Hwa Kim Jinhyun Jang Kwanghoon Sohn VLM 101 19 0 03 Apr 2023
Borrowing Human Senses: Comment-Aware Self-Training for Social Media Multimodal Classification Chunpu Xu Jing Li VLM 59 5 0 27 Mar 2023
Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey Tianlin Li Guangyao Chen Guangwu Qian Pengcheng Gao Xiaoyong Wei Yaowei Wang Yonghong Tian Wen Gao AI4CE VLM 151 214 0 20 Feb 2023
Multi-modal Machine Learning in Engineering Design: A Review and Future Directions Binyang Song Ruilin Zhou Faez Ahmed AI4CE 144 46 0 14 Feb 2023
A survey on knowledge-enhanced multimodal learning Maria Lymperaiou Giorgos Stamou 159 15 0 19 Nov 2022
Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information Weijie Su Xizhou Zhu Chenxin Tao Lewei Lu Bin Li Gao Huang Yu Qiao Xiaogang Wang Jie Zhou Jifeng Dai 97 42 0 17 Nov 2022
DiMBERT: Learning Vision-Language Grounded Representations with Disentangled Multimodal-Attention Fenglin Liu Xian Wu Shen Ge Xuancheng Ren Wei Fan Xu Sun Yuexian Zou VLM 108 13 0 28 Oct 2022
Masked Vision-Language Transformer in Fashion Ge-Peng Ji Mingchen Zhuge D. Gao Deng-Ping Fan Daniel Gehrig Luc Van Gool 90 25 0 27 Oct 2022
Learning by Hallucinating: Vision-Language Pre-training with Weak Supervision Tong Wang Jorma T. Laaksonen T. Langer Heikki Arponen Tom E. Bishop VLM 45 6 0 24 Oct 2022
Contrastive Language-Image Pre-Training with Knowledge Graphs Xuran Pan Tianzhu Ye Dongchen Han S. Song Gao Huang VLM CLIP 79 54 0 17 Oct 2022
Learning to Evaluate Performance of Multi-modal Semantic Localization Zhiqiang Yuan Wenkai Zhang Chongyang Li Zhaoying Pan Yongqiang Mao Jialiang Chen Shuoke Li Hongqi Wang Xian Sun 99 20 0 14 Sep 2022
Computational Sarcasm Analysis on Social Media: A Systematic Review Faria Binte Kader Nafisa Hossain Nujat Tasmia Binte Sogir Mohsinul Kabir H. Mahmud Md. Kamrul Hasan 50 5 0 13 Sep 2022
PreSTU: Pre-Training for Scene-Text Understanding Jihyung Kil Soravit Changpinyo Xi Chen Hexiang Hu Sebastian Goodman Wei-Lun Chao Radu Soricut VLM 191 29 0 12 Sep 2022
A Sketch Is Worth a Thousand Words: Image Retrieval with Text and Sketch Patsorn Sangkloy Wittawat Jitkrittum Diyi Yang James Hays 3DV 82 32 0 05 Aug 2022
Vision-and-Language Pretraining Thong Nguyen Cong-Duy Nguyen Xiaobao Wu See-Kiong Ng Anh Tuan Luu VLM CLIP 65 2 0 05 Jul 2022
Multimodal Learning with Transformers: A Survey Peng Xu Xiatian Zhu David Clifton ViT 236 575 0 13 Jun 2022
PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models Yuan Yao Qi-An Chen Ao Zhang Wei Ji Zhiyuan Liu Tat-Seng Chua Maosong Sun VLM MLLM 93 38 0 23 May 2022
Learning to Answer Visual Questions from Web Videos Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid ViT 89 35 0 10 May 2022
Training and challenging models for text-guided fashion image retrieval Eric Dodds Jack Culpepper Gaurav Srivastava 65 9 0 23 Apr 2022
Towards Lightweight Transformer via Group-wise Transformation for Vision-and-Language Tasks Gen Luo Yiyi Zhou Xiaoshuai Sun Yan Wang Liujuan Cao Yongjian Wu Feiyue Huang Rongrong Ji ViT 64 47 0 16 Apr 2022
Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration Xiwen Liang Fengda Zhu Lingling Li Hang Xu Xiaodan Liang LM&Ro VLM 58 30 0 08 Mar 2022
Vision-Language Intelligence: Tasks, Representation Learning, and Large Models Feng Li Hao Zhang Yi-Fan Zhang Shixuan Liu Jian Guo L. Ni Pengchuan Zhang Lei Zhang AI4TS VLM 79 37 0 03 Mar 2022
VLP: A Survey on Vision-Language Pre-training Feilong Chen Duzhen Zhang Minglun Han Xiuyi Chen Jing Shi Shuang Xu Bo Xu VLM 183 227 0 18 Feb 2022
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound Rowan Zellers Jiasen Lu Ximing Lu Youngjae Yu Yanpeng Zhao Mohammadreza Salehi Aditya Kusupati Jack Hessel Ali Farhadi Yejin Choi 113 215 0 07 Jan 2022
LaTr: Layout-Aware Transformer for Scene-Text VQA Ali Furkan Biten Ron Litman Yusheng Xie Srikar Appalaraju R. Manmatha ViT 125 102 0 23 Dec 2021
Decompose the Sounds and Pixels, Recompose the Events Varshanth R. Rao Md Ibrahim Khalil Haoda Li Peng Dai Juwei Lu 51 5 0 21 Dec 2021
Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Tasks Xizhou Zhu Jinguo Zhu Hao Li Xiaoshi Wu Xiaogang Wang Hongsheng Li Xiaohua Wang Jifeng Dai 124 133 0 02 Dec 2021
UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Faisal Ahmed Zicheng Liu Yumao Lu Lijuan Wang 146 117 0 23 Nov 2021
LAViTeR: Learning Aligned Visual and Textual Representations Assisted by Image and Caption Generation Mohammad Abuzar Shaikh Zhanghexuan Ji Dana Moukheiber Yan Shen S. Srihari Mingchen Gao VLM 44 1 0 04 Sep 2021
Audio-Visual Transformer Based Crowd Counting Usman Sajid Xiangyu Chen Hasan Sajid Taejoon Kim Guanghui Wang ViT 98 22 0 04 Sep 2021
Auto-Parsing Network for Image Captioning and Visual Question Answering Xu Yang Chongyang Gao Hanwang Zhang Jianfei Cai 117 37 0 24 Aug 2021
From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network Yuxin Wang Hongtao Xie Shancheng Fang Jing Wang Shenggao Zhu Yongdong Zhang VLM 94 154 0 22 Aug 2021
Airbert: In-domain Pretraining for Vision-and-Language Navigation Pierre-Louis Guhur Makarand Tapaswi Shizhe Chen Ivan Laptev Cordelia Schmid LM&Ro 59 144 0 20 Aug 2021
Knowledge Perceived Multi-modal Pretraining in E-commerce Yushan Zhu Huaixiao Tou Wen Zhang Ganqiang Ye Hui Chen Ningyu Zhang Huajun Chen 92 33 0 20 Aug 2021
Exceeding the Limits of Visual-Linguistic Multi-Task Learning Cameron R. Wolfe Keld T. Lundgaard VLM 76 2 0 27 Jul 2021
Multi-stage Pre-training over Simplified Multimodal Pre-training Models Tongtong Liu Fangxiang Feng Xiaojie Wang 38 13 0 22 Jul 2021