Polos: Multimodal Metric Learning from Human Feedback for Image Captioning

28 February 2024

Papers citing "Polos: Multimodal Metric Learning from Human Feedback for Image Captioning"

31 / 31 papers shown

Title
A Video-grounded Dialogue Dataset and Metric for Event-driven Activities Wiradee Imrattanatrai Masaki Asada Kimihiro Hasegawa Zhi-Qi Cheng Ken Fukuda Teruko Mitamura VGen 103 0 0 30 Jan 2025
Mobile Manipulation Instruction Generation from Multiple Images with Automatic Metric Enhancement Kei Katsumata Motonari Kambara Daichi Yashima Ryosuke Korekata Komei Sugiura 148 0 0 28 Jan 2025
SESCORE2: Learning Text Generation Evaluation via Synthesizing Realistic Mistakes Wenda Xu Xian Qian Mingxuan Wang Lei Li William Yang Wang 33 10 0 19 Dec 2022
Scaling Instruction-Finetuned Language Models Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay ... Jacob Devlin Adam Roberts Denny Zhou Quoc V. Le Jason W. Wei ReLM LRM 167 3,110 0 20 Oct 2022
Not All Errors are Equal: Learning Text Generation Metrics using Stratified Error Synthesis Wenda Xu Yi-Lin Tuan Yujie Lu Michael Stephen Saxon Lei Li William Yang Wang 79 22 0 10 Oct 2022
GIT: A Generative Image-to-text Transformer for Vision and Language Jianfeng Wang Zhengyuan Yang Xiaowei Hu Linjie Li Kevin Qinghong Lin Zhe Gan Zicheng Liu Ce Liu Lijuan Wang VLM 123 548 0 27 May 2022
Flamingo: a Visual Language Model for Few-Shot Learning Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr ... Mikolaj Binkowski Ricardo Barreira Oriol Vinyals Andrew Zisserman Karen Simonyan MLLM VLM 344 3,532 0 29 Apr 2022
OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework Peng Wang An Yang Rui Men Junyang Lin Shuai Bai Zhikang Li Jianxin Ma Chang Zhou Jingren Zhou Hongxia Yang MLLM ObjD 135 873 0 07 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong Guosheng Lin MLLM BDL VLM CLIP 501 4,324 0 28 Jan 2022
RegionCLIP: Region-based Language-Image Pretraining Yiwu Zhong Jianwei Yang Pengchuan Zhang Chunyuan Li Noel Codella ... Luowei Zhou Xiyang Dai Lu Yuan Yin Li Jianfeng Gao VLM CLIP 126 575 0 16 Dec 2021
CIDEr-R: Robust Consensus-based Image Description Evaluation G. O. D. Santos Esther Luna Colombini Sandra Avila 54 30 0 28 Sep 2021
From Show to Tell: A Survey on Deep Learning-based Image Captioning Matteo Stefanini Marcella Cornia Lorenzo Baraldi S. Cascianelli G. Fiameni Rita Cucchiara 3DV VLM MLLM 109 267 0 14 Jul 2021
Case Relation Transformer: A Crossmodal Language Generation Model for Fetching Instructions Motonari Kambara K. Sugiura ViT 51 6 0 02 Jul 2021
BARTScore: Evaluating Generated Text as Text Generation Weizhe Yuan Graham Neubig Pengfei Liu 95 841 0 22 Jun 2021
Multi-Modal Image Captioning for the Visually Impaired Hiba Ahsan Nikita Bhalla Daivat Bhatt Kaivankumar Shah 55 22 0 17 May 2021
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation Xiuye Gu Nayeon Lee Weicheng Kuo Huayu Chen VLM ObjD 263 915 0 28 Apr 2021
SimCSE: Simple Contrastive Learning of Sentence Embeddings Tianyu Gao Xingcheng Yao Danqi Chen AILaw SSL 253 3,371 0 18 Apr 2021
CLIPScore: A Reference-free Evaluation Metric for Image Captioning Jack Hessel Ari Holtzman Maxwell Forbes Ronan Le Bras Yejin Choi CLIP 117 1,545 0 18 Apr 2021
TextCaps: a Dataset for Image Captioning with Reading Comprehension Oleksii Sidorov Ronghang Hu Marcus Rohrbach Amanpreet Singh 58 411 0 24 Mar 2020
Captioning Images Taken by People Who Are Blind Danna Gurari Yinan Zhao Meng Zhang Nilavra Bhattacharya 59 182 0 20 Feb 2020
Meshed-Memory Transformer for Image Captioning Marcella Cornia Matteo Stefanini Lorenzo Baraldi Rita Cucchiara 59 879 0 17 Dec 2019
Unsupervised Cross-lingual Representation Learning at Scale Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzmán Edouard Grave Myle Ott Luke Zettlemoyer Veselin Stoyanov 195 6,538 0 05 Nov 2019
Attention on Attention for Image Captioning Lun Huang Wenmin Wang Jie Chen Xiao-Yong Wei 58 831 0 19 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 538 24,351 0 26 Jul 2019
Image Captioning: Transforming Objects into Words Simão Herdade Armin Kappeler K. Boakye Joao Soares ViT 101 469 0 14 Jun 2019
Learning to Evaluate Image Captioning Huayu Chen Guandao Yang Andreas Veit Xun Huang Serge J. Belongie 62 148 0 17 Jun 2018
Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering Peter Anderson Xiaodong He Chris Buehler Damien Teney Mark Johnson Stephen Gould Lei Zhang AIMat 113 4,208 0 25 Jul 2017
SPICE: Semantic Propositional Image Caption Evaluation Peter Anderson Basura Fernando Mark Johnson Stephen Gould EGVM 90 1,914 0 29 Jul 2016
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention Ke Xu Jimmy Ba Ryan Kiros Kyunghyun Cho Aaron Courville Ruslan Salakhutdinov R. Zemel Yoshua Bengio DiffM 328 10,050 0 10 Feb 2015
CIDEr: Consensus-based Image Description Evaluation Ramakrishna Vedantam C. L. Zitnick Devi Parikh 268 4,471 0 20 Nov 2014
Microsoft COCO: Common Objects in Context Nayeon Lee Michael Maire Serge J. Belongie Lubomir Bourdev Ross B. Girshick James Hays Pietro Perona Deva Ramanan C. L. Zitnick Piotr Dollár ObjD 387 43,619 0 01 May 2014