BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models

17 April 2021

Papers citing "BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models"

50 / 572 papers shown

Title
A Fresh Take on Stale Embeddings: Improving Dense Retriever Training with Corrector Networks Nicholas Monath Will Grathwohl Michael Boratko Rob Fergus Andrew McCallum Manzil Zaheer 33 0 0 03 Sep 2024
Know When to Fuse: Investigating Non-English Hybrid Retrieval in the Legal Domain Antoine Louis Gijs van Dijck Gerasimos Spanakis 36 0 0 02 Sep 2024
rerankers: A Lightweight Python Library to Unify Ranking Methods Benjamin Clavié ALM 40 6 0 30 Aug 2024
Rethinking Sparse Lexical Representations for Image Retrieval in the Age of Rising Multi-Modal Large Language Models K. Nakata Daisuke Miyashita Youyang Ng Yasuto Hoshi J. Deguchi 42 0 0 29 Aug 2024
Large Language Models as Foundations for Next-Gen Dense Retrieval: A Comprehensive Empirical Assessment Kun Luo Minghao Qin Zheng Liu Shitao Xiao Jun Zhao Kang Liu 42 10 0 22 Aug 2024
The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design Artem Snegirev Maria Tikhonova Anna Maksimova Alena Fenogenova Alexander Abramov 34 4 0 22 Aug 2024
Mistral-SPLADE: LLMs for better Learned Sparse Retrieval Meet Doshi Vishwajeet Kumar Rudra Murthy Vignesh P Jaydeep Sen RALM 35 2 0 20 Aug 2024
Task-level Distributionally Robust Optimization for Large Language Model-based Dense Retrieval Guangyuan Ma Yongliang Ma Xing Wu Zhenpeng Su Ming Zhou Songlin Hu OOD 41 2 0 20 Aug 2024
PhysBERT: A Text Embedding Model for Physics Scientific Literature Thorsten Hellert Joao Montenegro Andrea Pollastro PINN AI4CE 30 4 0 18 Aug 2024
Hindi-BEIR : A Large Scale Retrieval Benchmark in Hindi Arkadeep Acharya Rudra Murthy Vishwajeet Kumar Jaydeep Sen 45 1 0 18 Aug 2024
Constructing Domain-Specific Evaluation Sets for LLM-as-a-judge Ravi Raju Swayambhoo Jain Bo Li Jonathan Li Urmish Thakker ALM ELM 44 11 0 16 Aug 2024
RealMedQA: A pilot biomedical question answering dataset containing realistic clinical questions Gregory Kell A. Roberts Serge Umansky Yuti Khare Najma Ahmed ... Chloe Simela Jack Coumbe Julian Rozario Ryan-Rhys Griffiths Iain J. Marshall 49 0 0 16 Aug 2024
On the Element-Wise Representation and Reasoning in Zero-Shot Image Recognition: A Systematic Survey Jingcai Guo Zhijie Rao Zhi Chen Song Guo Jingren Zhou Dacheng Tao 36 3 0 09 Aug 2024
Learning to Rewrite: Generalized LLM-Generated Text Detection Wei Hao Ran Li Weiliang Zhao Junfeng Yang Chengzhi Mao DeLMO 59 3 0 08 Aug 2024
Developing PUGG for Polish: A Modern Approach to KBQA, MRC, and IR Dataset Construction Albert Sawczyn Katsiaryna Viarenich Konrad Wojtasik Aleksandra Domogała Marcin Oleksy Maciej Piasecki Tomasz Kajdanowicz 39 0 0 05 Aug 2024
Generative Retrieval with Few-shot Indexing Arian Askari Chuan Meng Mohammad Aliannejadi Zhaochun Ren Evangelos Kanoulas Suzan Verberne RALM 55 3 0 04 Aug 2024
An Encoding--Searching Separation Perspective on Bi-Encoder Neural Search Danbinaerin Han Akiko Aizawa Sihun Lee 29 0 0 02 Aug 2024
FiCo-ITR: bridging fine-grained and coarse-grained image-text retrieval for comparative performance analysis Mikel Williams-Lekuona Georgina Cosma 54 0 0 29 Jul 2024
Overview of PerpectiveArg2024: The First Shared Task on Perspective Argument Retrieval Neele Falk Andreas Waldis Iryna Gurevych LRM 27 5 0 29 Jul 2024
mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval Xin Zhang Yanzhao Zhang Dingkun Long Wen Xie Ziqi Dai ... Pengjun Xie Fei Huang Meishan Zhang Wenjie Li Min Zhang 42 78 0 29 Jul 2024
Banyan: Improved Representation Learning with Explicit Structure Mattia Opper N. Siddharth 31 1 0 25 Jul 2024
Enhancing LLM's Cognition via Structurization Kai-Chun Liu Zhihang Fu Chao Chen Wei Zhang Rongxin Jiang Fan Zhou Yao-Shen Chen Yue-bo Wu Jieping Ye 58 1 0 23 Jul 2024
Benchmarks as Microscopes: A Call for Model Metrology Michael Stephen Saxon Ari Holtzman Peter West William Y. Wang Naomi Saphra 39 10 0 22 Jul 2024
NV-Retriever: Improving text embedding models with effective hard-negative mining Gabriel de Souza P. Moreira Radek Osmulski Mengyao Xu Ronay Ak Benedikt Schifferer Even Oldridge RALM 49 31 0 22 Jul 2024
INDIC QA BENCHMARK: A Multilingual Benchmark to Evaluate Question Answering capability of LLMs for Indic Languages A. Singh Rudra Murthy Vishwajeet Kumar Jaydeep Sen Ashish Mittal Ganesh Ramakrishnan 45 6 0 18 Jul 2024
Matryoshka-Adaptor: Unsupervised and Supervised Tuning for Smaller Embedding Dimensions Jinsung Yoon Raj Sinha Sercan Ö. Arik Tomas Pfister 24 1 0 17 Jul 2024
Crafting the Path: Robust Query Rewriting for Information Retrieval Ingeol Baek Jimin Lee Joonho Yang Hwanhee Lee 49 5 0 17 Jul 2024
MSEval: A Dataset for Material Selection in Conceptual Design to Evaluate Algorithmic Models Yash Jain Daniele Grandi Allin Groom Brandon Cramer Christopher McComb 44 0 0 12 Jul 2024
Deep Bag-of-Words Model: An Efficient and Interpretable Relevance Architecture for Chinese E-Commerce Zhe Lin Jiwei Tan Dan Ou Xi Chen Shaowei Yao Bo Zheng 26 3 0 12 Jul 2024
LitSearch: A Retrieval Benchmark for Scientific Literature Search Anirudh Ajith Mengzhou Xia Alexis Chevalier Tanya Goyal Danqi Chen Tianyu Gao RALM 36 10 0 10 Jul 2024
Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective Yu-An Liu Ruqing Zhang Jiafeng Guo Maarten de Rijke Yixing Fan Xueqi Cheng 40 6 0 09 Jul 2024
BM25S: Orders of magnitude faster lexical search via eager sparse scoring Xing Han Lù 25 32 0 04 Jul 2024
CoIR: A Comprehensive Benchmark for Code Information Retrieval Models Xiangyang Li Kuicai Dong Yi Quan Lee Wei Xia Yichun Yin Xinyi Dai Yasheng Wang Ruiming Tang 65 15 0 03 Jul 2024
RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs Yue Yu Ming-Yu Liu Zihan Liu Wei Ping Jiaxuan You Chao Zhang M. Shoeybi Bryan Catanzaro ALM RALM 39 50 0 02 Jul 2024
Reliable Confidence Intervals for Information Retrieval Evaluation Using Generative A.I Harrie Oosterhuis R. Jagerman Zhen Qin Xuanhui Wang Michael Bendersky 59 4 0 02 Jul 2024
Searching for Best Practices in Retrieval-Augmented Generation Xiaohua Wang Zhenghua Wang Xuan Gao Feiran Zhang Yixin Wu ... Qi Qian Ruicheng Yin Changze Lv Xiaoqing Zheng Xuanjing Huang 60 41 0 01 Jul 2024
BERGEN: A Benchmarking Library for Retrieval-Augmented Generation David Rau Hervé Déjean Nadezhda Chirkova Thibault Formal Shuai Wang Vassilina Nikoulina S. Clinchant 45 12 0 01 Jul 2024
From RAG to RICHES: Retrieval Interlaced with Sequence Generation Palak Jain Livio Baldini Soares Tom Kwiatkowski VLM 29 4 0 29 Jun 2024
ColPali: Efficient Document Retrieval with Vision Language Models Manuel Faysse Hugues Sibille Tony Wu Bilel Omrani Gautier Viaud C´eline Hudelot Pierre Colombo VLM 70 21 0 27 Jun 2024
Re-Ranking Step by Step: Investigating Pre-Filtering for Re-Ranking with Large Language Models Baharan Nouriinanloo Maxime Lamothe 45 0 0 26 Jun 2024
"Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Models Zhen Tan Chengshuai Zhao Raha Moraffah Yifan Li Song Wang Jundong Li Tianlong Chen Huan Liu SILM 54 18 0 26 Jun 2024
Efficient Document Ranking with Learnable Late Interactions Ziwei Ji Himanshu Jain Andreas Veit Sashank J. Reddi Sadeep Jayasumana A. S. Rawat A. Menon Felix X. Yu Sanjiv Kumar 46 1 0 25 Jun 2024
X-ray Made Simple: Lay Radiology Report Generation and Robust Evaluation Kun Zhao Chenghao Xiao Chen Tang Bohao Yang Kai Ye Noura Al Moubayed Liang Zhan Chenghua Lin 53 0 0 25 Jun 2024
CLERC: A Dataset for Legal Case Retrieval and Retrieval-Augmented Analysis Generation Abe Bohan Hou Orion Weller Guanghui Qin Eugene Yang Dawn J Lawrie Nils Holzenberger Andrew Blair-Stanek Benjamin Van Durme AILaw ELM 87 6 0 24 Jun 2024
DEXTER: A Benchmark for open-domain Complex Question Answering using LLMs Venktesh V. Deepali Prabhu Avishek Anand RALM CoGe 39 3 0 24 Jun 2024
Ragnarök: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Track Ronak Pradeep Nandan Thakur Sahel Sharifymoghaddam Eric Zhang Ryan Nguyen Daniel Campos Nick Craswell Jimmy Lin 40 12 0 24 Jun 2024
Database-Augmented Query Representation for Information Retrieval Soyeong Jeong Jinheon Baek Sukmin Cho Sung Ju Hwang Jong C. Park 42 2 0 23 Jun 2024
News Deja Vu: Connecting Past and Present with Semantic Search Brevin Franklin Emily Silcock Abhishek Arora Tom Bryan Melissa Dell 36 1 0 21 Jun 2024
Leveraging Passage Embeddings for Efficient Listwise Reranking with Large Language Models Qi Liu Bo Wang Nan Wang Jiaxin Mao RALM 80 3 0 21 Jun 2024
RE-AdaptIR: Improving Information Retrieval through Reverse Engineered Adaptation William Fleshman Benjamin Van Durme 48 0 0 20 Jun 2024