Comprehending and Ordering Semantics for Image Captioning

14 June 2022

Yehao Li

Yingwei Pan

Ting Yao

Tao Mei

ArXiv PDF HTML

Papers citing "Comprehending and Ordering Semantics for Image Captioning"

36 / 36 papers shown

Title
Image Captioning Evaluation in the Age of Multimodal LLMs: Challenges and Future Perspectives Sara Sarto Marcella Cornia Rita Cucchiara 46 0 0 18 Mar 2025
RONA: Pragmatically Diverse Image Captioning with Coherence Relations Aashish Anantha Ramakrishnan Aadarsh Anantha Ramakrishnan Dongwon Lee 56 0 0 14 Mar 2025
Unleashing Text-to-Image Diffusion Prior for Zero-Shot Image Captioning Jianjie Luo Jingwen Chen Yehao Li Yingwei Pan Jianlin Feng Hongyang Chao Ting Yao DiffM VLM 53 0 0 03 Jan 2025
Phrase Decoupling Cross-Modal Hierarchical Matching and Progressive Position Correction for Visual Grounding Minghong Xie Hao Wu Huafeng Li Yafei Zhang Dapeng Tao Z. Yu ObjD 40 1 0 31 Oct 2024
Positive-Augmented Contrastive Learning for Vision-and-Language Evaluation and Training Sara Sarto Nicholas Moratelli Marcella Cornia Lorenzo Baraldi Rita Cucchiara 42 3 0 09 Oct 2024
Revisiting Image Captioning Training Paradigm via Direct CLIP-based Optimization Nicholas Moratelli Davide Caffagni Marcella Cornia Lorenzo Baraldi Rita Cucchiara CLIP 31 3 0 26 Aug 2024
Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends and Metrics Analysis Uri Berger Gabriel Stanovsky Omri Abend Lea Frermann 35 0 0 09 Aug 2024
BRIDGE: Bridging Gaps in Image Captioning Evaluation with Stronger Visual Cues Sara Sarto Marcella Cornia Lorenzo Baraldi Rita Cucchiara 38 6 0 29 Jul 2024
Mitigating Open-Vocabulary Caption Hallucinations Assaf Ben-Kish Moran Yanuka Morris Alper Raja Giryes Hadar Averbuch-Elor MLLM VLM 23 6 0 06 Dec 2023
Improving Image Captioning via Predicting Structured Concepts Ting Wang Weidong Chen Yuanhe Tian Yan Song Zhendong Mao 31 8 0 14 Nov 2023
Bounding and Filling: A Fast and Flexible Framework for Image Captioning Zheng Ma Changxin Wang Bo Huang Zi-Yue Zhu Jianbing Zhang 23 1 0 15 Oct 2023
Rewrite Caption Semantics: Bridging Semantic Gaps for Language-Supervised Semantic Segmentation Yun Xing Jian Kang Aoran Xiao Jiahao Nie Ling Shao Shijian Lu VLM 38 12 0 24 Sep 2023
With a Little Help from your own Past: Prototypical Memory Networks for Image Captioning Manuele Barraco Sara Sarto Marcella Cornia Lorenzo Baraldi Rita Cucchiara VLM 55 19 0 23 Aug 2023
CgT-GAN: CLIP-guided Text GAN for Image Captioning Jiarui Yu Haoran Li Y. Hao B. Zhu Tong Xu Xiangnan He VLM CLIP 21 13 0 23 Aug 2023
Artificial-Spiking Hierarchical Networks for Vision-Language Representation Learning Ye-Ting Chen Siyu Zhang Yaoru Sun Weijian Liang Haoran Wang 40 0 0 18 Aug 2023
Simple Token-Level Confidence Improves Caption Correctness Suzanne Petryk Spencer Whitehead Joseph E. Gonzalez Trevor Darrell Anna Rohrbach Marcus Rohrbach 28 7 0 11 May 2023
Caption Anything: Interactive Image Description with Diverse Multimodal Controls Teng Wang Jinrui Zhang Junjie Fei Hao Zheng Yunlong Tang Zhe Li Mingqi Gao Shanshan Zhao MLLM 102 82 0 04 May 2023
Fine-grained Audible Video Description Xuyang Shen Dong Li Jinxing Zhou Zhen Qin Bowen He ... Yuchao Dai Lingpeng Kong Meng Wang Yu Qiao Yiran Zhong VGen 36 11 0 27 Mar 2023
Selectively Hard Negative Mining for Alleviating Gradient Vanishing in Image-Text Matching Zheng Li Caili Guo Xin Wang Zerun Feng Zhongtian Du VLM 18 4 0 01 Mar 2023
CREPE: Can Vision-Language Foundation Models Reason Compositionally? Zixian Ma Jerry Hong Mustafa Omer Gul Mona Gandhi Irena Gao Ranjay Krishna CoGe 29 125 0 13 Dec 2022
Semantic-Conditional Diffusion Networks for Image Captioning Jianjie Luo Yehao Li Yingwei Pan Ting Yao Jianlin Feng Hongyang Chao Tao Mei DiffM 22 62 0 06 Dec 2022
Uncertainty-Aware Image Captioning Zhengcong Fei Mingyuan Fan Li Zhu Junshi Huang Xiaoming Wei Xiaolin K. Wei UQLM 18 10 0 30 Nov 2022
Retrieval-Augmented Multimodal Language Modeling Michihiro Yasunaga Armen Aghajanyan Weijia Shi Rich James J. Leskovec Percy Liang M. Lewis Luke Zettlemoyer Wen-tau Yih RALM 13 95 0 22 Nov 2022
Exploring Discrete Diffusion Models for Image Captioning Zixin Zhu Yixuan Wei Jianfeng Wang Zhe Gan Zheng-Wei Zhang Le Wang G. Hua Lijuan Wang Zicheng Liu Han Hu DiffM VLM 23 17 0 21 Nov 2022
Vision Transformer Based Model for Describing a Set of Images as a Story Zainy M. Malakan Ghulam Mubashar Hassan Ajmal Saeed Mian ViT 23 6 0 06 Oct 2022
Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning Ting Yao Yingwei Pan Yehao Li Chong-Wah Ngo Tao Mei ViT 151 137 0 11 Jul 2022
Dual Vision Transformer Ting Yao Yehao Li Yingwei Pan Yu Wang Xiaoping Zhang Tao Mei ViT 141 75 0 11 Jul 2022
Silver-Bullet-3D at ManiSkill 2021: Learning-from-Demonstrations and Heuristic Rule-based Methods for Object Manipulation Yingwei Pan Yehao Li Yiheng Zhang Qi Cai Fuchen Long Zhaofan Qiu Ting Yao Tao Mei 15 3 0 13 Jun 2022
Prompt-based Learning for Unpaired Image Captioning Peipei Zhu Tianlin Li Lin Zhu Zhenglong Sun Weishi Zheng Yaowei Wang C. L. P. Chen VLM 23 31 0 26 May 2022
Deep Learning Approaches on Image Captioning: A Review Taraneh Ghandi H. Pourreza H. Mahyar VLM 16 89 0 31 Jan 2022
Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets Marcella Cornia Lorenzo Baraldi G. Fiameni Rita Cucchiara 20 12 0 24 Nov 2021
How Much Can CLIP Benefit Vision-and-Language Tasks? Sheng Shen Liunian Harold Li Hao Tan Joey Tianyi Zhou Anna Rohrbach Kai-Wei Chang Z. Yao Kurt Keutzer CLIP VLM MLLM 196 405 0 13 Jul 2021
Auto-captions on GIF: A Large-scale Video-sentence Dataset for Vision-language Pre-training Yingwei Pan Yehao Li Jianjie Luo Jun Xu Ting Yao Tao Mei 30 57 0 05 Jul 2020
Normalized and Geometry-Aware Self-Attention Network for Image Captioning Longteng Guo Jing Liu Xinxin Zhu Peng Yao Shichen Lu Hanqing Lu ViT 120 189 0 19 Mar 2020
Neural Baby Talk Jiasen Lu Jianwei Yang Dhruv Batra Devi Parikh VLM 200 434 0 27 Mar 2018
Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning Jiasen Lu Caiming Xiong Devi Parikh R. Socher 85 1,442 0 06 Dec 2016