Benchmarking Large Language Models for News Summarization

31 January 2023

Tianyi Zhang

Faisal Ladhak

Esin Durmus

Percy Liang

Kathleen McKeown

Tatsunori B. Hashimoto

ELM

ArXiv (abs)PDF HTML

Papers citing "Benchmarking Large Language Models for News Summarization"

50 / 54 papers shown

Title
ECHO-LLaMA: Efficient Caching for High-Performance LLaMA Training Maryam Dialameh Rezaul Karim Hossein Rajabzadeh Omar Mohamed Awad Hyock Ju Kwon Boxing Chen Walid Ahmed Yang Liu 76 0 0 22 May 2025
Enhancing Abstractive Summarization of Scientific Papers Using Structure Information Tong Bao Heng Zhang Chengzhi Zhang 199 3 0 20 May 2025
R3: Robust Rubric-Agnostic Reward Models David Anugraha Zilu Tang Lester James V. Miranda Hanyang Zhao Mohammad Rifqi Farhansyah Garry Kuwanto Derry Wijaya Genta Indra Winata 176 1 0 19 May 2025
Multi2: Multi-Agent Test-Time Scalable Framework for Multi-Document Processing Juntai Cao Xiang Zhang Raymond Li Chuyuan Li Shafiq Joty Shafiq Joty Giuseppe Carenini 152 1 0 27 Feb 2025
Neural Attention Search Difan Deng Marius Lindauer 137 0 0 21 Feb 2025
Valuable Hallucinations: Realizable Non-realistic Propositions Qiucheng Chen Bo Wang LRM 124 0 0 16 Feb 2025
Evaluating Small Language Models for News Summarization: Implications and Factors Influencing Performance Borui Xu Yao Chen Zeyi Wen Weiguo Liu Bingsheng He 164 2 0 02 Feb 2025
Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models Jingwei Yi Yueqi Xie Bin Zhu Emre Kiciman Guangzhong Sun Xing Xie Fangzhao Wu AAML 136 82 0 28 Jan 2025
Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer Inference Weizhi Fei Xueyan Niu Guoqing Xie Yingqing Liu Bo Bai Wei Han 120 1 0 22 Jan 2025
Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation Yuhui Zhang Yuchang Su Yiming Liu Xiaohan Wang James Burgess ... Josiah Aklilu Alejandro Lozano Anjiang Wei Ludwig Schmidt Serena Yeung-Levy 135 5 0 06 Jan 2025
Zero-Shot Strategies for Length-Controllable Summarization Fabian Retkowski A. Waibel 115 4 0 31 Dec 2024
Modeling Story Expectations to Understand Engagement: A Generative Framework Using LLMs Hortense Fong George Gui HAI 128 0 0 13 Dec 2024
Coverage-based Fairness in Multi-document Summarization Haoyuan Li Yusen Zhang Rui Zhang Snigdha Chaturvedi 159 0 0 11 Dec 2024
Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset Khaoula Chehbouni Jonathan Colaço-Carr Yash More Jackie CK Cheung G. Farnadi 160 1 0 12 Nov 2024
Beemo: Benchmark of Expert-edited Machine-generated Outputs Ekaterina Artemova Jason Samuel Lucas Saranya Venkatraman Jooyoung Lee Sergei Tilga Adaku Uchendu Vladislav Mikhailov DeLMO MoE 140 8 0 06 Nov 2024
Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting Can Chen Jun-Kun Wang DeLMO 131 0 0 29 Oct 2024
DiscoGraMS: Enhancing Movie Screen-Play Summarization using Movie Character-Aware Discourse Graph Maitreya Prafulla Chitale Uday Bindal Rajakrishnan Rajkumar Rahul Mishra 80 1 0 18 Oct 2024
HSR-Enhanced Sparse Attention Acceleration Bo Chen Yingyu Liang Zhizhou Sha Zhenmei Shi Zhao Song 215 22 0 14 Oct 2024
MetaMetrics: Calibrating Metrics For Generation Tasks Using Human Preferences Genta Indra Winata David Anugraha Lucky Susanto Garry Kuwanto Derry Wijaya 122 11 0 03 Oct 2024
ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understanding Yijia Xiao Edward Sun Yiqiao Jin Qifan Wang Wei Wang 92 15 0 21 Aug 2024
Confidence-weighted integration of human and machine judgments for superior decision-making Felipe Yánez Xiaoliang Luo Omar Valerio Minero Bradley C. Love 102 2 0 15 Aug 2024
ThinK: Thinner Key Cache by Query-Driven Pruning Yuhui Xu Zhanming Jie Hanze Dong Lei Wang Xudong Lu Aojun Zhou Amrita Saha Caiming Xiong Doyen Sahoo 153 22 0 30 Jul 2024
Cascading Large Language Models for Salient Event Graph Generation Xingwei Tan Yuxiang Zhou Gabriele Pergola Yulan He 102 1 0 26 Jun 2024
PlagBench: Exploring the Duality of Large Language Models in Plagiarism Generation and Detection Jooyoung Lee Toshini Agrawal Adaku Uchendu Thai V. Le Jinghui Chen Dongwon Lee 164 1 0 24 Jun 2024
LLM Reading Tea Leaves: Automatically Evaluating Topic Models with Large Language Models Xiaohao Yang He Zhao Dinh Q. Phung Wray Buntine Lan Du ALM ELM 139 2 0 13 Jun 2024
Hallucination of Multimodal Large Language Models: A Survey Zechen Bai Pichao Wang Tianjun Xiao Tong He Zongbo Han Zheng Zhang Mike Zheng Shou VLM LRM 204 195 0 29 Apr 2024
A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods Hanlei Jin Yang Zhang Dan Meng Jun Wang Jinghua Tan 237 96 0 05 Mar 2024
GhostWriter: Augmenting Collaborative Human-AI Writing Experiences Through Personalization and Agency Catherine Yeh Gonzalo A. Ramos Rachel Ng Andy Huntington Richard Banks LLMAG 93 22 0 13 Feb 2024
Natural Language Reinforcement Learning Xidong Feng Bo Liu Mengyue Yang Ziyan Wang Girish A. Koushiks Yali Du Ying Wen Jun Wang OffRL 89 5 0 11 Feb 2024
Spurious Correlations in Reference-Free Evaluation of Text Generation Esin Durmus Faisal Ladhak Tatsunori Hashimoto 62 31 0 21 Apr 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 529 6,293 0 05 Apr 2022
BRIO: Bringing Order to Abstractive Summarization Yixin Liu Pengfei Liu Dragomir R. Radev Graham Neubig 96 285 0 31 Mar 2022
SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization Philippe Laban Tobias Schnabel Paul N. Bennett Marti A. Hearst HILM 107 396 0 18 Nov 2021
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 355 1,708 0 15 Oct 2021
Recursively Summarizing Books with Human Feedback Jeff Wu Long Ouyang Daniel M. Ziegler Nissan Stiennon Ryan J. Lowe Jan Leike Paul Christiano ALM 177 303 0 22 Sep 2021
BARTScore: Evaluating Generated Text as Text Generation Weizhe Yuan Graham Neubig Pengfei Liu 123 849 0 22 Jun 2021
GLM: General Language Model Pretraining with Autoregressive Blank Infilling Zhengxiao Du Yujie Qian Xiao Liu Ming Ding J. Qiu Zhilin Yang Jie Tang BDL AI4CE 142 1,553 0 18 Mar 2021
Evaluating Factuality in Generation with Dependency-level Entailment Tanya Goyal Greg Durrett 114 151 0 12 Oct 2020
FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization Esin Durmus He He Mona T. Diab HILM 93 396 0 07 May 2020
On Faithfulness and Factuality in Abstractive Summarization Joshua Maynez Shashi Narayan Bernd Bohnet Ryan T. McDonald HILM 86 1,039 0 02 May 2020
Improved Natural Language Generation via Loss Truncation Daniel Kang Tatsunori Hashimoto 70 97 0 30 Apr 2020
BLEU might be Guilty but References are not Innocent Markus Freitag David Grangier Isaac Caswell 57 149 0 13 Apr 2020
Fill in the BLANC: Human-free quality estimation of document summaries Oleg V. Vasilyev Vedant Dharnidharka John Bohannon 3DH 83 119 0 23 Feb 2020
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization Jingqing Zhang Yao-Min Zhao Mohammad Saleh Peter J. Liu RALM 3DGS 297 2,051 0 18 Dec 2019
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 474 1,766 0 18 Sep 2019
Unified Language Model Pre-training for Natural Language Understanding and Generation Li Dong Nan Yang Wenhui Wang Furu Wei Xiaodong Liu Yu Wang Jianfeng Gao M. Zhou H. Hon ELM AI4CE 227 1,560 0 08 May 2019
BERTScore: Evaluating Text Generation with BERT Tianyi Zhang Varsha Kishore Felix Wu Kilian Q. Weinberger Yoav Artzi 352 5,860 0 21 Apr 2019
Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization Shashi Narayan Shay B. Cohen Mirella Lapata AILaw 146 1,683 0 27 Aug 2018
Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies Max Grusky Mor Naaman Yoav Artzi 93 558 0 30 Apr 2018
Get To The Point: Summarization with Pointer-Generator Networks A. See Peter J. Liu Christopher D. Manning 3DPC 311 4,026 0 14 Apr 2017