Dwell in the Beginning: How Language Models Embed Long Documents for Dense Retrieval

5 April 2024

Papers citing "Dwell in the Beginning: How Language Models Embed Long Documents for Dense Retrieval"

7 / 7 papers shown

Title
Benchmarking and Building Long-Context Retrieval Models with LoCo and M2-BERT Jon Saad-Falcon Daniel Y. Fu Simran Arora Neel Guha Christopher Ré RALM 34 16 0 12 Feb 2024
Making Large Language Models A Better Foundation For Dense Retrieval Chaofan Li Zheng Liu Shitao Xiao Yingxia Shao RALM 34 34 0 24 Dec 2023
RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder Shitao Xiao Zheng Liu Yingxia Shao Zhao Cao RALM 118 109 0 24 May 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 253 695 0 27 Aug 2021
Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval Luyu Gao Jamie Callan RALM 169 329 0 12 Aug 2021
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models Nandan Thakur Nils Reimers Andreas Rucklé Abhishek Srivastava Iryna Gurevych VLM 231 971 0 17 Apr 2021
Pretrained Transformers for Text Ranking: BERT and Beyond Jimmy J. Lin Rodrigo Nogueira Andrew Yates VLM 239 611 0 13 Oct 2020