Moving Beyond Downstream Task Accuracy for Information Retrieval Benchmarking

2 December 2022

Papers citing "Moving Beyond Downstream Task Accuracy for Information Retrieval Benchmarking"

21 / 21 papers shown

Title
MIRAGE-Bench: Automatic Multilingual Benchmark Arena for Retrieval-Augmented Generation Systems Nandan Thakur Suleman Kazi Ge Luo Jimmy J. Lin Amin Ahmad VLM RALM 130 7 0 17 Oct 2024
CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval Minghan Li Sheng-Chieh Lin Barlas Oğuz Asish Ghoshal Jimmy J. Lin Yashar Mehdad Wen-tau Yih Xilun Chen 41 26 0 18 Nov 2022
Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages Xinyu Crystina Zhang Nandan Thakur Odunayo Ogundepo Ehsan Kamalloo David Alfonso-Hermelo Xiaoguang Li Qun Liu Mehdi Rezagholizadeh Jimmy J. Lin 114 54 0 18 Oct 2022
An Efficiency Study for SPLADE Models Carlos Lassance Stéphane Clinchant 71 70 0 08 Jul 2022
PLAID: An Efficient Engine for Late Interaction Retrieval Keshav Santhanam Omar Khattab Christopher Potts Matei A. Zaharia VLM 79 76 0 19 May 2022
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction Keshav Santhanam Omar Khattab Jon Saad-Falcon Christopher Potts Matei A. Zaharia 73 405 0 02 Dec 2021
RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking Ruiyang Ren Yingqi Qu Jing Liu Wayne Xin Zhao Qiaoqiao She Hua Wu Haifeng Wang Ji-Rong Wen 231 255 0 14 Oct 2021
Towards Efficient NLP: A Standard Evaluation and A Strong Baseline Xiangyang Liu Tianxiang Sun Junliang He Jiawen Wu Lingling Wu Xinyu Zhang Hao Jiang Bo Zhao Xuanjing Huang Xipeng Qiu ELM 54 47 0 13 Oct 2021
Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval Xinyu Crystina Zhang Xueguang Ma Peng Shi Jimmy J. Lin 61 114 0 19 Aug 2021
Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval Luyu Gao Jamie Callan RALM 248 336 0 12 Aug 2021
Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking Zhiyi Ma Kawin Ethayarajh Tristan Thrush Somya Jain Ledell Yu Wu Robin Jia Christopher Potts Adina Williams Douwe Kiela ELM 69 57 0 21 May 2021
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models Nandan Thakur Nils Reimers Andreas Rucklé Abhishek Srivastava Iryna Gurevych VLM 391 1,030 0 17 Apr 2021
ExplainaBoard: An Explainable Leaderboard for NLP Pengfei Liu Jinlan Fu Yanghua Xiao Weizhe Yuan Shuaichen Chang Junqi Dai Yixin Liu Zihuiwen Ye Zi-Yi Dou Graham Neubig XAI LRM ELM 64 55 0 13 Apr 2021
Traditional IR rivals neural models on the MS MARCO Document Ranking Leaderboard Leonid Boytsov VLM 13 4 0 15 Dec 2020
XOR QA: Cross-lingual Open-Retrieval Question Answering Akari Asai Jungo Kasai J. Clark Kenton Lee Eunsol Choi Hannaneh Hajishirzi 91 150 0 22 Oct 2020
Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval Lee Xiong Chenyan Xiong Ye Li Kwok-Fung Tang Jialin Liu Paul N. Bennett Junaid Ahmed Arnold Overwijk 107 1,218 0 01 Jul 2020
ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT Omar Khattab Matei A. Zaharia 115 1,356 0 27 Apr 2020
Dense Passage Retrieval for Open-Domain Question Answering Vladimir Karpukhin Barlas Oğuz Sewon Min Patrick Lewis Ledell Yu Wu Sergey Edunov Danqi Chen Wen-tau Yih RALM 154 3,739 0 10 Apr 2020
MLPerf Training Benchmark Arya D. McCarthy Christine Cheng Cody Coleman Greg Diamos Paulius Micikevicius ... Carole-Jean Wu Lingjie Xu Masafumi Yamazaki C. Young Matei A. Zaharia 84 311 0 02 Oct 2019
Billion-scale similarity search with GPUs Jeff Johnson Matthijs Douze Hervé Jégou 220 3,712 0 28 Feb 2017
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset Payal Bajaj Daniel Fernando Campos Nick Craswell Li Deng Jianfeng Gao ... Mir Rosenberg Xia Song Alina Stoica Saurabh Tiwary Tong Wang RALM 127 2,721 0 28 Nov 2016