SciRepEval: A Multi-Format Benchmark for Scientific Document Representations

23 November 2022

Amanpreet Singh

Arman Cohan

Papers citing "SciRepEval: A Multi-Format Benchmark for Scientific Document Representations"

50 / 52 papers shown

Title
How Deep Do Large Language Models Internalize Scientific Literature and Citation Practices? Andres Algaba Vincent Holst Floriano Tori Melika Mobini Brecht Verbeken Sylvia Wenmackers Vincent Ginis 41 1 0 03 Apr 2025
Towards AI-assisted Academic Writing Daniel J. Liebling Malcolm Kane Madeleine Grunde-Mclaughlin Ian J. Lang Subhashini Venugopalan Michael P. Brenner 58 0 0 17 Mar 2025
CorpusStudio: Surfacing Emergent Patterns in a Corpus of Prior Work while Writing Hai Dang Chelse Swoopes Daniel Buschek Elena L. Glassman 44 0 0 16 Mar 2025
Improving Scientific Document Retrieval with Concept Coverage-based Query Set Generation SeongKu Kang Bowen Jin Wonbin Kweon Yu Zhang Dongha Lee J. Han Hwanjo Yu 61 0 0 16 Feb 2025
Automatic Evaluation Metrics for Artificially Generated Scientific Research Niklas Höpner Leon Eshuijs Dimitrios Alivanistos Giacomo Zamprogno Ilaria Tiddi 49 0 0 14 Feb 2025
Efficient Scientific Full Text Classification: The Case of EICAT Impact Assessments Marc Felix Brinner Sina Zarrieß 84 1 0 10 Feb 2025
Indonesian-English Code-Switching Speech Synthesizer Utilizing Multilingual STEN-TTS and Bert LID Ahmad Alfani Handoyo Chung Tran Dessi Lestari S. Sakti 28 0 0 26 Dec 2024
Taxonomy-guided Semantic Indexing for Academic Paper Search SeongKu Kang Yunyi Zhang Pengcheng Jiang Dongha Lee J. Han Hwanjo Yu 34 3 0 25 Oct 2024
HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction Qianyue Hao Jingyang Fan Fengli Xu Jian Yuan Yong Li 29 0 0 10 Oct 2024
Conversational Exploratory Search of Scholarly Publications Using Knowledge Graphs Phillip Schneider Florian Matthes 21 0 0 01 Oct 2024
An Investigation of Warning Erroneous Chat Translations in Cross-lingual Communication Yunmeng Li Jun Suzuki Makoto Morishita Kaori Abe Kentaro Inui 65 1 0 28 Aug 2024
The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design Artem Snegirev Maria Tikhonova Anna Maksimova Alena Fenogenova Alexander Abramov 28 4 0 22 Aug 2024
WeQA: A Benchmark for Retrieval Augmented Generation in Wind Energy Domain Rounak Meyur Hung Phan S. Wagle Jan Strube M. Halappanavar Sameera Horawalavithana Anurag Acharya Sai Munikoti 25 1 0 21 Aug 2024
PhysBERT: A Text Embedding Model for Physics Scientific Literature Thorsten Hellert Joao Montenegro Andrea Pollastro PINN AI4CE 22 4 0 18 Aug 2024
CHIME: LLM-Assisted Hierarchical Organization of Scientific Studies for Literature Review Support Chao-Chun Hsu Erin Bransom Jenna Sparks Bailey Kuehl Chenhao Tan David Wadden Lucy Lu Wang Aakanksha Naik 33 6 0 23 Jul 2024
HDT: Hierarchical Document Transformer Haoyu He Markus Flicke Jan Buchmann Iryna Gurevych Andreas Geiger 40 0 0 11 Jul 2024
CiteME: Can Language Models Accurately Cite Scientific Claims? Ori Press Andreas Hochlehnert Ameya Prabhu Vishaal Udandarao Ofir Press Matthias Bethge 44 13 0 10 Jul 2024
NLP-KG: A System for Exploratory Search of Scientific Literature in Natural Language Processing Tim Schopf Florian Matthes 57 0 0 21 Jun 2024
A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery Yu Zhang Xiusi Chen Bowen Jin Sheng Wang Shuiwang Ji Wei Wang Jiawei Han 44 27 0 16 Jun 2024
CancerLLM: A Large Language Model in Cancer Domain Mingchen Li Jiatan Huang Jeremy Yeung A. Blaes Steven Johnson Hongfang Liu Hua Xu Rui Zhang ELM LM&MA 32 4 0 15 Jun 2024
SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literature David Wadden Kejian Shi Jacob Morrison Aakanksha Naik Shruti Singh ... Luca Soldaini Shannon Zejiang Shen Doug Downey Hannaneh Hajishirzi Arman Cohan 47 11 0 10 Jun 2024
Enriched BERT Embeddings for Scholarly Publication Classification Benjamin Wolff Eva Seidlmayer Konrad U. Förstner 21 3 0 07 May 2024
TOPICAL: TOPIC Pages AutomagicaLly John Giorgi Amanpreet Singh Doug Downey Sergey Feldman Lucy Lu Wang MedIm 34 0 0 03 May 2024
BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers Ran Xu Wenqi Shi Yue Yu Yuchen Zhuang Yanqiao Zhu M. D. Wang Joyce C. Ho Chao Zhang Carl Yang LM&MA 40 19 0 29 Apr 2024
LLMs4OM: Matching Ontologies with Large Language Models Hamed Babaei Giglou Jennifer D'Souza Felix Engel Sören Auer 40 9 0 16 Apr 2024
On-the-fly Definition Augmentation of LLMs for Biomedical NER Monica Munnangi Sergey Feldman Byron C. Wallace Silvio Amir Tom Hope Aakanksha Naik 32 11 0 29 Mar 2024
Acceleron: A Tool to Accelerate Research Ideation Harshit Nigam Manasi S. Patwardhan L. Vig Gautam M. Shroff 38 6 0 07 Mar 2024
PaperWeaver: Enriching Topical Paper Alerts by Contextualizing Recommended Papers with User-collected Papers Yoonjoo Lee Hyeonsu B Kang Matt Latzke Juho Kim Jonathan Bragg Joseph Chee Chang Pao Siangliulue 21 18 0 05 Mar 2024
PaECTER: Patent-level Representation Learning using Citation-informed Transformers Mainak Ghosh Sebastian Erhardt Michael E. Rose Erik Buunk Dietmar Harhoff 21 2 0 29 Feb 2024
BIRCO: A Benchmark of Information Retrieval Tasks with Complex Objectives Xiaoyue Wang Jianyou Wang Weili Cao Kaicheng Wang R. Paturi Leon Bergen 37 6 0 21 Feb 2024
Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning Shivalika Singh Freddie Vargus Daniel D'souza Börje F. Karlsson Abinaya Mahendiran ... Max Bartolo Julia Kreutzer A. Ustun Marzieh Fadaee Sara Hooker 119 117 0 09 Feb 2024
Tweets to Citations: Unveiling the Impact of Social Media Influencers on AI Research Visibility Iain Xie Weissburg Mehir Arora Xinyi Wang Liangming Pan Luu Anh Tuan 21 3 0 24 Jan 2024
PaperQA: Retrieval-Augmented Generative Agent for Scientific Research Jakub Lála Odhran O'Donoghue Aleksandar Shtedritski Sam Cox Samuel G. Rodriques Andrew D. White RALM 75 73 0 08 Dec 2023
ATLANTIC: Structure-Aware Retrieval-Augmented Language Model for Interdisciplinary Science Sai Munikoti Anurag Acharya S. Wagle Sameera Horawalavithana RALM 41 7 0 21 Nov 2023
CARE: Extracting Experimental Findings From Clinical Literature Aakanksha Naik Bailey Kuehl Erin Bransom Doug Downey Tom Hope 28 1 0 16 Nov 2023
Personalized Jargon Identification for Enhanced Interdisciplinary Communication Yue Guo Joseph Chee Chang Maria Antoniak Erin Bransom Trevor Cohen Lucy Lu Wang Tal August 15 10 0 16 Nov 2023
Evaluating the Effectiveness of Retrieval-Augmented Large Language Models in Scientific Document Reasoning Sai Munikoti Anurag Acharya S. Wagle Sameera Horawalavithana LRM 19 5 0 07 Nov 2023
NuclearQA: A Human-Made Benchmark for Language Models for the Nuclear Domain Anurag Acharya Sai Munikoti Aaron Hellinger Sara Smith S. Wagle Sameera Horawalavithana ELM 25 4 0 17 Oct 2023
When Large Language Models Meet Citation: A Survey Yang Zhang Yufei Wang Kai Wang Quan Z. Sheng Lina Yao A. Mahmood Wei Emma Zhang Rongying Zhao 50 7 0 18 Sep 2023
MultiSChuBERT: Effective Multimodal Fusion for Scholarly Document Quality Prediction Gideon Maillette de Buy Wenniger Thomas van Dongen Lambert Schomaker 13 4 0 15 Aug 2023
Exploring the Landscape of Natural Language Processing Research Tim Schopf Karim Arabi Florian Matthes 21 13 0 20 Jul 2023
ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews Mike DÁrcy Alexis Ross Erin Bransom Bailey Kuehl Jonathan Bragg Tom Hope Doug Downey KELM 24 21 0 21 Jun 2023
SynerGPT: In-Context Learning for Personalized Drug Synergy Prediction and Drug Design Carl N. Edwards Aakanksha Naik Tushar Khot Martin Burke Heng Ji Tom Hope 28 14 0 19 Jun 2023
Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding Yu Zhang Hao Cheng Zhihong Shen Xiaodong Liu Yejiang Wang Jianfeng Gao 29 13 0 23 May 2023
A Gold Standard Dataset for the Reviewer Assignment Problem Ivan Stelmakh John Wieting Graham Neubig Nihar B. Shah 22 15 0 23 Mar 2023
Assisting Human Decisions in Document Matching Joon Sik Kim Valerie Chen Danish Pruthi Nihar B. Shah Ameet Talwalkar 40 5 0 16 Feb 2023
The Semantic Scholar Open Data Platform Rodney Michael Kinney Chloe Anastasiades Russell Authur Iz Beltagy Jonathan Bragg ... Caroline M Wu Jiangjiang Yang Angele Zamarron Madeleine van Zuylen Daniel S. Weld 16 93 0 24 Jan 2023
Transfer learning with affine model transformation Shunya Minami Kenji Fukumizu Yoshihiro Hayashi Ryo Yoshida 24 1 0 18 Oct 2022
Text and Code Embeddings by Contrastive Pre-Training Arvind Neelakantan Tao Xu Raul Puri Alec Radford Jesse Michael Han ... Tabarak Khan Toki Sherbakov Joanne Jang Peter Welinder Lilian Weng SSL AI4TS 218 422 0 24 Jan 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,657 0 15 Oct 2021