Title
SCCA: Shifted Cross Chunk Attention for long contextual semantic expansion Yuxiang Guo 17 0 0 12 Dec 2023
Hashmarks: Privacy-Preserving Benchmarks for High-Stakes AI Evaluation P. Bricman 19 0 0 01 Dec 2023
ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up? Hailin Chen Fangkai Jiao Xingxuan Li Chengwei Qin Mathieu Ravaut Ruochen Zhao Caiming Xiong Shafiq R. Joty ELM CLL AI4MH LRM ALM 85 27 0 28 Nov 2023
LongBoX: Evaluating Transformers on Long-Sequence Clinical Tasks Mihir Parmar Aakanksha Naik Himanshu Gupta Disha Agrawal Chitta Baral LM&MA 22 3 0 16 Nov 2023
MAVEN-Arg: Completing the Puzzle of All-in-One Event Understanding Dataset with Event Argument Annotation Xiaozhi Wang Hao Peng Yong Guan Kaisheng Zeng Jianhui Chen ... Yankai Lin Zhiyuan Liu Ruobing Xie Jie Zhou Juanzi Li 21 4 0 15 Nov 2023
When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks Hao Peng Xiaozhi Wang Jianhui Chen Weikai Li Y. Qi ... Zhili Wu Kaisheng Zeng Bin Xu Lei Hou Juanzi Li 31 28 0 15 Nov 2023
LooGLE: Can Long-Context Language Models Understand Long Contexts? Jiaqi Li Mengmeng Wang Zilong Zheng Muhan Zhang ELM RALM 32 107 0 08 Nov 2023
M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Models Wai-Chung Kwan Xingshan Zeng Yufei Wang Yusen Sun Liangyou Li Lifeng Shang Qun Liu Kam-Fai Wong ELM 94 10 0 30 Oct 2023
Extending Input Contexts of Language Models through Training on Segmented Sequences Petros Karypis Julian McAuley George Karypis 29 0 0 23 Oct 2023
AutoMix: Automatically Mixing Language Models Pranjal Aggarwal Aman Madaan Ankit Anand Srividya Pranavi Potharaju Swaroop Mishra ... Karthik Kappaganthu Yiming Yang Shyam Upadhyay Manaal Faruqui Mausam 42 17 0 19 Oct 2023
In-context Pretraining: Language Modeling Beyond Document Boundaries Weijia Shi Sewon Min Maria Lomeli Chunting Zhou Margaret Li ... Victoria Lin Noah A. Smith Luke Zettlemoyer Scott Yih Mike Lewis LRM RALM SyDa 34 48 0 16 Oct 2023
On Context Utilization in Summarization with Large Language Models Mathieu Ravaut Aixin Sun Nancy F. Chen Shafiq R. Joty 33 13 0 16 Oct 2023
Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading Howard Chen Ramakanth Pasunuru Jason Weston Asli Celikyilmaz RALM 68 72 0 08 Oct 2023
Retrieval meets Long Context Large Language Models Peng-Tao Xu Ming-Yu Liu Xianchao Wu Lawrence C. McAfee Chen Zhu Zihan Liu Sandeep Subramanian Evelina Bakhturina M. Shoeybi Bryan Catanzaro RALM LRM 14 79 0 04 Oct 2023
BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models Zican Dong Tianyi Tang Junyi Li Wayne Xin Zhao Ji-Rong Wen RALM ALM 28 33 0 23 Sep 2023
XGen-7B Technical Report Erik Nijkamp Tian Xie Hiroaki Hayashi Bo Pang Congying Xia ... Chien-Sheng Wu Silvio Savarese Yingbo Zhou Shafiq R. Joty Caiming Xiong ALM 26 13 0 07 Sep 2023
FArMARe: a Furniture-Aware Multi-task methodology for Recommending Apartments based on the user interests Ali Abdari Alex Falcon Giuseppe Serra 32 2 0 06 Sep 2023
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding Yushi Bai Xin Lv Jiajie Zhang Hong Lyu Jiankai Tang ... Aohan Zeng Lei Hou Yuxiao Dong Jie Tang Juanzi Li LLMAG RALM 31 496 0 28 Aug 2023
Chunk, Align, Select: A Simple Long-sequence Processing Method for Transformers Jiawen Xie Pengyu Cheng Xiao Liang Yong Dai Nan Du 40 7 0 25 Aug 2023
L-Eval: Instituting Standardized Evaluation for Long Context Language Models Chen An Shansan Gong Ming Zhong Xingjian Zhao Mukai Li Jun Zhang Lingpeng Kong Xipeng Qiu ELM ALM 40 132 0 20 Jul 2023
Llama 2: Open Foundation and Fine-Tuned Chat Models Hugo Touvron Louis Martin Kevin R. Stone Peter Albert Amjad Almahairi ... Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov Thomas Scialom AI4MH ALM 105 11,007 0 18 Jul 2023
Retentive Network: A Successor to Transformer for Large Language Models Yutao Sun Li Dong Shaohan Huang Shuming Ma Yuqing Xia Jilong Xue Jianyong Wang Furu Wei LRM 78 301 0 17 Jul 2023
Lost in the Middle: How Language Models Use Long Contexts Nelson F. Liu Kevin Lin John Hewitt Ashwin Paranjape Michele Bevilacqua Fabio Petroni Percy Liang RALM 40 1,404 0 06 Jul 2023
Extending Context Window of Large Language Models via Positional Interpolation Shouyuan Chen Sherman Wong Liangjian Chen Yuandong Tian 12 494 0 27 Jun 2023
Long Text Generation Challenge N. Mikhaylovskiy ALM 16 0 0 04 Jun 2023
PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents Simeng Sun Y. Liu Shuohang Wang Chenguang Zhu Mohit Iyyer RALM LRM ReLM 30 51 0 23 May 2023
ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding Uri Shaham Maor Ivgi Avia Efrat Jonathan Berant Omer Levy VLM 38 126 0 23 May 2023
NarrativeXL: A Large-scale Dataset For Long-Term Memory Models A. Moskvichev Ky-Vinh Mai RALM 13 1 0 23 May 2023
VNHSGE: VietNamese High School Graduation Examination Dataset for Large Language Models Dao Xuan-Quy Le Ngoc-Bich Vo The-Duy Phan Xuan-Dung Ngo Bac-Bien Nguyen Van-Tien Nguyen Thi-My-Thanh Nguyen Hong-Phuoc 19 16 0 20 May 2023
Prefix Propagation: Parameter-Efficient Tuning for Long Sequences Jonathan Li Will Aitken R. Bhambhoria Xiao-Dan Zhu 17 14 0 20 May 2023
DLUE: Benchmarking Document Language Understanding Ruoxi Xu Hongyu Lin Xinyan Guan Xianpei Han Yingfei Sun Le Sun ELM 24 0 0 16 May 2023
Vcc: Scaling Transformers to 128K Tokens or More by Prioritizing Important Tokens Zhanpeng Zeng Cole Hawkins Min-Fong Hong Aston Zhang Nikolaos Pappas Vikas Singh Shuai Zheng 21 6 0 07 May 2023
Unlimiformer: Long-Range Transformers with Unlimited Length Input Amanda Bertsch Uri Alon Graham Neubig Matthew R. Gormley RALM 99 122 0 02 May 2023
CoLT5: Faster Long-Range Transformers with Conditional Computation Joshua Ainslie Tao Lei Michiel de Jong Santiago Ontañón Siddhartha Brahma ... Mandy Guo James Lee-Thorp Yi Tay Yun-hsuan Sung Sumit Sanghai LLMAG 31 63 0 17 Mar 2023
MAUD: An Expert-Annotated Legal NLP Dataset for Merger Agreement Understanding Steven H. Wang Antoine Scardigli Leonard Tang Wei Chen D.M. Levkin Anya Chen Spencer Ball Thomas Woodside Oliver Zhang Dan Hendrycks AILaw ELM 30 16 0 02 Jan 2023
Parallel Context Windows for Large Language Models Nir Ratner Yoav Levine Yonatan Belinkov Ori Ram Inbal Magar Omri Abend Ehud D. Karpas Amnon Shashua Kevin Leyton-Brown Y. Shoham RALM 17 69 0 21 Dec 2022
A Length-Extrapolatable Transformer Yutao Sun Li Dong Barun Patra Shuming Ma Shaohan Huang Alon Benhaim Vishrav Chaudhary Xia Song Furu Wei 30 115 0 20 Dec 2022
Pretraining Without Attention Junxiong Wang J. Yan Albert Gu Alexander M. Rush 27 48 0 20 Dec 2022
LawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrieval William F. Bruno Dan Roth ELM AILaw 25 6 0 06 Dec 2022
Efficient Transformers with Dynamic Token Pooling Piotr Nawrot J. Chorowski Adrian Lañcucki E. Ponti 20 42 0 17 Nov 2022
CREATIVESUMM: Shared Task on Automatic Summarization for Creative Writing Divyansh Agarwal Alexander R. Fabbri Simeng Han Wojciech Kry'sciñski Faisal Ladhak B. Li Kathleen McKeown Dragomir R. Radev Tianyi Zhang Sam Wiseman 23 5 0 10 Nov 2022
Language Models Understand Us, Poorly Jared Moore LRM 17 4 0 19 Oct 2022
What Makes Convolutional Models Great on Long Sequence Modeling? Yuhong Li Tianle Cai Yi Zhang De-huai Chen Debadeepta Dey VLM 36 96 0 17 Oct 2022
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling Jinchao Zhang Shuyang Jiang Jiangtao Feng Lin Zheng Lingpeng Kong 3DV 43 9 0 14 Oct 2022
Hierarchical3D Adapters for Long Video-to-text Summarization Pinelopi Papalampidi Mirella Lapata VGen 29 12 0 10 Oct 2022
Adapting Pretrained Text-to-Text Models for Long Text Sequences Wenhan Xiong Anchit Gupta Shubham Toshniwal Yashar Mehdad Wen-tau Yih RALM VLM 54 30 0 21 Sep 2022
Investigating Efficiently Extending Transformers for Long Input Summarization Jason Phang Yao-Min Zhao Peter J. Liu RALM LLMAG 29 63 0 08 Aug 2022
Efficient Long-Text Understanding with Short-Text Models Maor Ivgi Uri Shaham Jonathan Berant VLM 27 75 0 01 Aug 2022
Conditional Generation with a Question-Answering Blueprint Shashi Narayan Joshua Maynez Reinald Kim Amplayo Kuzman Ganchev Annie Louis Fantine Huot Anders Sandholm Dipanjan Das Mirella Lapata 57 47 0 01 Jul 2022
SQuALITY: Building a Long-Document Summarization Dataset the Hard Way Alex Jinpeng Wang Richard Yuanzhe Pang Angelica Chen Jason Phang Samuel R. Bowman 74 44 0 23 May 2022