BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models

17 April 2021

Papers citing "BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models"

50 / 572 papers shown

Title
Augmenting Query and Passage for Retrieval-Augmented Generation using LLMs for Open-Domain Question Answering Minsang Kim Cheoneum Park Seungjun Baek RALM 40 2 0 20 Jun 2024
An Investigation of Prompt Variations for Zero-shot LLM-based Rankers Shuoqi Sun Shengyao Zhuang Shuai Wang Guido Zuccon 42 5 0 20 Jun 2024
CodeRAG-Bench: Can Retrieval Augment Code Generation? Zora Zhiruo Wang Akari Asai Xinyan Velocity Yu Frank F. Xu Yiqing Xie Graham Neubig Daniel Fried RALM 80 30 0 20 Jun 2024
APEER: Automatic Prompt Engineering Enhances Large Language Model Reranking Can Jin Hongwu Peng Shiyu Zhao Zhenting Wang Wujiang Xu Ligong Han Jiahui Zhao Kai Zhong Sanguthevar Rajasekaran Dimitris N. Metaxas KELM 44 32 0 20 Jun 2024
Improving Zero-shot LLM Re-Ranker with Risk Minimization Xiaowei Yuan Zhao Yang Yequan Wang Jun Zhao Kang Liu 33 1 0 19 Jun 2024
Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? Jinhyuk Lee Anthony Chen Zhuyun Dai Dheeru Dua Devendra Singh Sachan ... Jeremy R. Cole Sebastian Riedel Iftekhar Naim Ming-Wei Chang Kelvin Guu RALM LRM 58 30 0 19 Jun 2024
PromptDSI: Prompt-based Rehearsal-free Instance-wise Incremental Learning for Document Retrieval Tuan-Luc Huynh Thuy-Trang Vu Weiqing Wang Yinwei Wei Trung Le D. Gašević Yuan-Fang Li Thanh-Toan Do VLM CLL 43 0 0 18 Jun 2024
TourRank: Utilizing Large Language Models for Documents Ranking with a Tournament-Inspired Strategy Yiqun Chen Qi Liu Yi Zhang Weiwei Sun Daiting Shi Jiaxin Mao Dawei Yin Jiaxin Mao Dawei Yin 33 6 0 17 Jun 2024
SparseCL: Sparse Contrastive Learning for Contradiction Retrieval Haike Xu Zongyu Lin Ningyu Zhang Kai-Wei Chang Piotr Indyk 45 0 0 15 Jun 2024
Can't Hide Behind the API: Stealing Black-Box Commercial Embedding Models Manveer Singh Tamber Jasper Xian Jimmy Lin MLAU SILM 175 0 0 13 Jun 2024
MINERS: Multilingual Language Models as Semantic Retrievers Genta Indra Winata Ruochen Zhang David Ifeoluwa Adelani RALM 49 5 0 11 Jun 2024
Measuring and Addressing Indexical Bias in Information Retrieval Caleb Ziems William B. Held Jane Dwivedi-Yu Diyi Yang 25 3 0 06 Jun 2024
Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe Alicja Ziarko Albert Q. Jiang Bartosz Piotrowski Wenda Li M. Jamnik Piotr Miłoś 40 0 0 06 Jun 2024
A Bi-metric Framework for Fast Similarity Search Haike Xu Sandeep Silwal Piotr Indyk FedML 31 0 0 05 Jun 2024
The Scandinavian Embedding Benchmarks: Comprehensive Assessment of Multilingual and Monolingual Text Embedding Kenneth C. Enevoldsen Márton Kardos Niklas Muennighoff Kristoffer Laigaard Nielbo 42 9 0 04 Jun 2024
BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models Jiaqi Xue Meng Zheng Yebowen Hu Fei Liu Xun Chen Qian Lou AAML SILM 38 26 0 03 Jun 2024
LLM-RankFusion: Mitigating Intrinsic Inconsistency in LLM-based Ranking Yifan Zeng Ojas Tendolkar Raymond Baartmans Qingyun Wu Huazheng Wang Lizhong Chen 39 0 0 31 May 2024
Passage-specific Prompt Tuning for Passage Reranking in Question Answering with Large Language Models Xuyang Wu Zhiyuan Peng Sravanthi Rajanala Hsin-Tai Wu Yi Fang LRM RALM 48 0 0 31 May 2024
MTEB-French: Resources for French Sentence Embedding Evaluation and Analysis Mathieu Ciancone Imene Kerboua Marion Schaeffer W. Siblini 42 2 0 30 May 2024
Generative Query Reformulation Using Ensemble Prompting, Document Fusion, and Relevance Feedback Kaustubh D. Dhole Ramraj Chandradevan Eugene Agichtein 21 1 0 27 May 2024
NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models Chankyu Lee Rajarshi Roy Mengyao Xu Jonathan Raiman M. Shoeybi Bryan Catanzaro Ming-Yu Liu RALM 68 145 0 27 May 2024
Cocktail: A Comprehensive Information Retrieval Benchmark with LLM-Generated Documents Integration Sunhao Dai Weihao Liu Yuqi Zhou Liang Pang Rongju Ruan Gang Wang Zhenhua Dong Jun Xu Jirong Wen 54 8 0 26 May 2024
Towards Completeness-Oriented Tool Retrieval for Large Language Models Changle Qu Sunhao Dai Xiaochi Wei Hengyi Cai Shuaiqiang Wang Dawei Yin Jun Xu Jirong Wen KELM 31 7 0 25 May 2024
AGRaME: Any-Granularity Ranking with Multi-Vector Embeddings R. Reddy Omar Attia Yunyao Li Heng Ji Saloni Potdar 40 1 0 23 May 2024
TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Models Pengzhou Cheng Yidong Ding Tianjie Ju Zongru Wu Wei Du Ping Yi Zhuosheng Zhang Gongshen Liu SILM AAML 40 20 0 22 May 2024
Question-Based Retrieval using Atomic Units for Enterprise RAG Vatsal Raina Mark J. F. Gales 35 7 0 20 May 2024
INDUS: Effective and Efficient Language Models for Scientific Applications Bishwaranjan Bhattacharjee Aashka Trivedi Masayasu Muraoka Muthukumaran Ramasubramanian Takuma Udagawa ... Peter W. J. Staar S. Vahidinia Ryan McGranaghan A. Mehrabian Tsendgar Lee AI4CE 33 5 0 17 May 2024
PL-MTEB: Polish Massive Text Embedding Benchmark Rafal Po'swiata Slawomir Dadas Michal Perelkiewicz 40 4 0 16 May 2024
Synthetic Test Collections for Retrieval Evaluation Hossein A. Rahmani Nick Craswell Emine Yilmaz Bhaskar Mitra Daniel Fernando Campos 23 18 0 13 May 2024
ViWikiFC: Fact-Checking for Vietnamese Wikipedia-Based Textual Knowledge Source Hung Tuan Le Long Truong To Manh Trong Nguyen Kiet Van Nguyen 43 3 0 13 May 2024
Explaining Text Similarity in Transformer Models Alexandros Vasileiou Oliver Eberle 43 7 0 10 May 2024
Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Models Luke Merrick Danmei Xu Gaurav Nuti Daniel Campos 17 24 0 08 May 2024
Adaptive Retrieval and Scalable Indexing for k-NN Search with Cross-Encoders Nishant Yadav Nicholas Monath Manzil Zaheer Rob Fergus Andrew McCallum CML RALM 40 1 0 06 May 2024
Beyond Relevance: Evaluate and Improve Retrievers on Perspective Awareness Xinran Zhao Tong Chen Sihao Chen Hongming Zhang Tongshuang Wu 34 7 0 04 May 2024
When to Retrieve: Teaching LLMs to Utilize Information Retrieval Effectively Tiziano Labruna Jon Ander Campos Gorka Azkune 26 10 0 30 Apr 2024
BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers Ran Xu Wenqi Shi Yue Yu Yuchen Zhuang Yanqiao Zhu M. D. Wang Joyce C. Ho Chao Zhang Carl Yang LM&MA 40 19 0 29 Apr 2024
URL: Universal Referential Knowledge Linking via Task-instructed Representation Compression Zhuoqun Li Hongyu Lin Tianshu Wang Boxi Cao Yaojie Lu Weixiang Zhou Hao Wang Zhenyu Zeng Le Sun Xianpei Han 51 1 0 24 Apr 2024
A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industry Yining Huang Keke Tang Meilian Chen Boyuan Wang ELM LM&MA 29 15 0 24 Apr 2024
Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations Sukmin Cho Soyeong Jeong Jeongyeon Seo Taeho Hwang Jong C. Park SILM AAML 52 27 0 22 Apr 2024
FASTTRACK: Fast and Accurate Fact Tracing for LLMs Si-An Chen Feiyang Kang Ning Yu Ruoxi Jia 32 0 0 22 Apr 2024
LongEmbed: Extending Embedding Models for Long Context Retrieval Dawei Zhu Liang Wang Nan Yang Yifan Song Wenhao Wu Furu Wei Sujian Li RALM 43 21 0 18 Apr 2024
AdvisorQA: Towards Helpful and Harmless Advice-seeking Question Answering with Collective Intelligence Minbeom Kim Hwanhee Lee Joonsuk Park Hwaran Lee Kyomin Jung 40 1 0 18 Apr 2024
Generalized Contrastive Learning for Multi-Modal Retrieval and Ranking Tianyu Zhu M. Jung Jesse Clark 91 1 0 12 Apr 2024
RAR-b: Reasoning as Retrieval Benchmark Chenghao Xiao G. Thomas Al Moubayed LRM RALM 36 8 0 09 Apr 2024
LLM-Augmented Retrieval: Enhancing Retrieval Models Through Language Models and Doc-Level Embedding Mingrui Wu Sheng Cao KELM RALM 43 3 0 08 Apr 2024
Q-PEFT: Query-dependent Parameter Efficient Fine-tuning for Text Reranking with Large Language Models Zhiyuan Peng Xuyang Wu Qifan Wang Sravanthi Rajanala Yi Fang 61 4 0 06 Apr 2024
Dwell in the Beginning: How Language Models Embed Long Documents for Dense Retrieval Joao Coelho Bruno Martins João Magalhães Jamie Callan Chenyan Xiong RALM 55 4 0 05 Apr 2024
GenQREnsemble: Zero-Shot LLM Ensemble Prompting for Generative Query Reformulation Kaustubh D. Dhole Eugene Agichtein 27 14 0 04 Apr 2024
DUQGen: Effective Unsupervised Domain Adaptation of Neural Rankers by Diversifying Synthetic Query Generation Ramraj Chandradevan Kaustubh D. Dhole Eugene Agichtein 48 3 0 03 Apr 2024
CLAPNQ: Cohesive Long-form Answers from Passages in Natural Questions for RAG systems Sara Rosenthal Avirup Sil Radu Florian Salim Roukos 56 11 0 02 Apr 2024