Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets

24 November 2021

Lorenzo Baraldi

Papers citing "Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets"

50 / 62 papers shown

Title
Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluation Sara Sarto Manuele Barraco Marcella Cornia Lorenzo Baraldi Rita Cucchiara 60 58 0 21 Mar 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 401 4,508 0 30 Jan 2023
LAION-5B: An open large-scale dataset for training next generation image-text models Christoph Schuhmann Romain Beaumont Richard Vencu Cade Gordon Ross Wightman ... Srivatsa Kundurthy Katherine Crowson Ludwig Schmidt R. Kaczmarczyk J. Jitsev VLM MLLM CLIP 134 3,420 0 16 Oct 2022
Retrieval-Augmented Transformer for Image Captioning Sara Sarto Marcella Cornia Lorenzo Baraldi Rita Cucchiara 60 58 0 26 Jul 2022
Comprehending and Ordering Semantics for Image Captioning Yehao Li Yingwei Pan Ting Yao Tao Mei 55 89 0 14 Jun 2022
CoCa: Contrastive Captioners are Image-Text Foundation Models Jiahui Yu Zirui Wang Vijay Vasudevan Legg Yeung Mojtaba Seyedhosseini Yonghui Wu VLM CLIP OffRL 131 1,293 0 04 May 2022
Diverse Image Captioning with Grounded Style Franz Klein Shweta Mahajan S. Roth 47 7 0 03 May 2022
OPT: Open Pre-trained Transformer Language Models Susan Zhang Stephen Roller Naman Goyal Mikel Artetxe Moya Chen ... Daniel Simig Punit Singh Koura Anjali Sridhar Tianlu Wang Luke Zettlemoyer VLM OSLM AI4CE 292 3,634 0 02 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 320 3,515 0 29 Apr 2022
CaMEL: Mean Teacher Learning for Image Captioning Manuele Barraco Matteo Stefanini Marcella Cornia S. Cascianelli Lorenzo Baraldi Rita Cucchiara ViT VLM 53 27 0 21 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 490 4,324 0 28 Jan 2022
Scaling Up Vision-Language Pre-training for Image Captioning Xiaowei Hu Zhe Gan Jianfeng Wang Zhengyuan Yang Zicheng Liu Yumao Lu Lijuan Wang MLLM VLM 112 248 0 24 Nov 2021
ClipCap: CLIP Prefix for Image Captioning Ron Mokady Amir Hertz Amit H. Bermano CLIP VLM 65 672 0 18 Nov 2021
LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs Christoph Schuhmann Richard Vencu Romain Beaumont R. Kaczmarczyk Clayton Mullis Aarush Katta Theo Coombes J. Jitsev Aran Komatsuzaki VLM MLLM CLIP 204 1,422 0 03 Nov 2021
Similar Scenes arouse Similar Emotions: Parallel Data Augmentation for Stylized Image Captioning Guodun Li Yuchen Zhai Zehao Lin Yin Zhang 81 21 0 26 Aug 2021
SimVLM: Simple Visual Language Model Pretraining with Weak Supervision Zirui Wang Jiahui Yu Adams Wei Yu Zihang Dai Yulia Tsvetkov Yuan Cao VLM MLLM 112 792 0 24 Aug 2021
Grid-VLP: Revisiting Grid Features for Vision-Language Pre-training Ming Yan Haiyang Xu Chenliang Li Bin Bi Junfeng Tian Min Gui Wei Wang VLM 53 10 0 21 Aug 2021
From Show to Tell: A Survey on Deep Learning-based Image Captioning Matteo Stefanini Marcella Cornia Lorenzo Baraldi S. Cascianelli G. Fiameni Rita Cucchiara 3DV VLM MLLM 106 267 0 14 Jul 2021
How Much Can CLIP Benefit Vision-and-Language Tasks? Sheng Shen Liunian Harold Li Hao Tan Joey Tianyi Zhou Anna Rohrbach Kai-Wei Chang Z. Yao Kurt Keutzer CLIP VLM MLLM 253 408 0 13 Jul 2021
E2E-VLP: End-to-End Vision-Language Pre-training Enhanced by Visual Learning Haiyang Xu Ming Yan Chenliang Li Bin Bi Songfang Huang Wenming Xiao Fei Huang VLM 55 119 0 03 Jun 2021
CLIPScore: A Reference-free Evaluation Metric for Image Captioning Jack Hessel Ari Holtzman Maxwell Forbes Ronan Le Bras Yejin Choi CLIP 117 1,545 0 18 Apr 2021
WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning Krishna Srinivasan K. Raman Jiecao Chen Michael Bendersky Marc Najork VLM 253 316 0 02 Mar 2021
Learning Transferable Visual Models From Natural Language Supervision Alec Radford Jong Wook Kim Chris Hallacy Aditya A. Ramesh Gabriel Goh ... Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger Ilya Sutskever CLIP VLM 808 29,167 0 26 Feb 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 426 1,120 0 17 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 417 3,826 0 11 Feb 2021
ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision Wonjae Kim Bokyung Son Ildoo Kim VLM CLIP 112 1,735 0 05 Feb 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 362 1,961 0 31 Dec 2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai ... Matthias Minderer G. Heigold Sylvain Gelly Jakob Uszkoreit N. Houlsby ViT 526 40,739 0 22 Oct 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 602 41,736 0 28 May 2020
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks Xiujun Li Xi Yin Chunyuan Li Pengchuan Zhang Xiaowei Hu ... Houdong Hu Li Dong Furu Wei Yejin Choi Jianfeng Gao VLM 88 1,934 0 13 Apr 2020
X-Linear Attention Networks for Image Captioning Yingwei Pan Ting Yao Yehao Li Tao Mei 92 510 0 31 Mar 2020
TextCaps: a Dataset for Image Captioning with Reading Comprehension Oleksii Sidorov Ronghang Hu Marcus Rohrbach Amanpreet Singh 58 411 0 24 Mar 2020
Captioning Images Taken by People Who Are Blind Danna Gurari Yinan Zhao Meng Zhang Nilavra Bhattacharya 59 182 0 20 Feb 2020
Meshed-Memory Transformer for Image Captioning Marcella Cornia Matteo Stefanini Lorenzo Baraldi Rita Cucchiara 59 874 0 17 Dec 2019
UNITER: UNiversal Image-TExt Representation Learning Yen-Chun Chen Linjie Li Licheng Yu Ahmed El Kholy Faisal Ahmed Zhe Gan Yu Cheng Jingjing Liu VLM OT 97 447 0 25 Sep 2019
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 337 934 0 24 Sep 2019
LXMERT: Learning Cross-Modality Encoder Representations from Transformers Hao Hao Tan Joey Tianyi Zhou VLM MLLM 227 2,474 0 20 Aug 2019
Attention on Attention for Image Captioning Lun Huang Wenmin Wang Jie Chen Xiao-Yong Wei 56 829 0 19 Aug 2019
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-training Gen Li Nan Duan Yuejian Fang Ming Gong Daxin Jiang Ming Zhou SSL VLM MLLM 198 900 0 16 Aug 2019
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks Jiasen Lu Dhruv Batra Devi Parikh Stefan Lee SSL VLM 217 3,667 0 06 Aug 2019
Aligning Linguistic Words and Visual Semantic Units for Image Captioning Longteng Guo Jing Liu Jinhui Tang Jiangwei Li W. Luo Hanqing Lu 47 102 0 06 Aug 2019
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes Yang You Jing Li Sashank J. Reddi Jonathan Hseu Sanjiv Kumar Srinadh Bhojanapalli Xiaodan Song J. Demmel Kurt Keutzer Cho-Jui Hsieh ODL 204 993 0 01 Apr 2019
nocaps: novel object captioning at scale Harsh Agrawal Karan Desai Yufei Wang Xinlei Chen Rishabh Jain Mark Johnson Dhruv Batra Devi Parikh Stefan Lee Peter Anderson VLM 102 476 0 20 Dec 2018
Auto-Encoding Scene Graphs for Image Captioning Xu Yang Kaihua Tang Hanwang Zhang Jianfei Cai 140 698 0 06 Dec 2018
The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale Alina Kuznetsova H. Rom N. Alldrin J. Uijlings Ivan Krasin ... S. Popov Matteo Malloci Alexander Kolesnikov Tom Duerig V. Ferrari ObjD VLM 96 1,345 0 02 Nov 2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.4K 94,511 0 11 Oct 2018
"Factual" or "Emotional": Stylized Image Captioning with Adaptive Learning and Attention Tianlang Chen Zhongping Zhang Quanzeng You Chen Fang Zhaowen Wang Hailin Jin Jiebo Luo 68 87 0 10 Jul 2018
SemStyle: Learning to Generate Stylised Image Captions using Unaligned Text A. Mathews Lexing Xie Xuming He VLM 61 115 0 18 May 2018
Mixed Precision Training Paulius Micikevicius Sharan Narang Jonah Alben G. Diamos Erich Elsen ... Boris Ginsburg Michael Houston Oleksii Kuchaiev Ganesh Venkatesh Hao Wu 149 1,792 0 10 Oct 2017
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering Peter Anderson Xiaodong He Chris Buehler Damien Teney Mark Johnson Stephen Gould Lei Zhang AIMat 111 4,208 0 25 Jul 2017