Benchmarking Large Language Models for News Summarization

31 January 2023

Tianyi Zhang

Faisal Ladhak

Esin Durmus

Percy Liang

Kathleen McKeown

Tatsunori B. Hashimoto

ELM

ArXiv PDF HTML

Papers citing "Benchmarking Large Language Models for News Summarization"

50 / 300 papers shown

Title
BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models Wei Qi Leong Jian Gang Ngui Yosephine Susanto Hamsawardhini Rengarajan Kengatharaiyer Sarveswaran William-Chandra Tjhi 26 9 0 12 Sep 2023
GenAIPABench: A Benchmark for Generative AI-based Privacy Assistants Aamir Hamid Hemanth Reddy Samidi Tim Finin Primal Pappachan Roberto Yus ELM 22 8 0 10 Sep 2023
FaNS: a Facet-based Narrative Similarity Metric Mousumi Akter Shubhra (Santu) Karmaker 25 1 0 09 Sep 2023
From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting Griffin Adams Alexander R. Fabbri Faisal Ladhak Eric Lehman Noémie Elhadad 29 52 0 08 Sep 2023
Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphs Chao Feng Xinyu Zhang Zichu Fei KELM 25 45 0 06 Sep 2023
Examining User-Friendly and Open-Sourced Large GPT Models: A Survey on Language, Multimodal, and Scientific GPT Models Kaiyuan Gao Su He Zhenyu He Jiacheng Lin Qizhi Pei Jie Shao Wei Zhang LM&MA SyDa 32 4 0 27 Aug 2023
KnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Bases Xintao Wang Qian Yang Yongting Qiu Jiaqing Liang Qi He Zhouhong Gu Yanghua Xiao Wei Wang KELM RALM 47 39 0 17 Aug 2023
Iterative Zero-Shot LLM Prompting for Knowledge Graph Construction S. Carta Alessandro Giuliani L. piano Alessandro Sebastian Podda Livio Pompianu Sandro Gabriele Tiddia 25 31 0 03 Jul 2023
Towards Measuring the Representation of Subjective Global Opinions in Language Models Esin Durmus Karina Nyugen Thomas I. Liao Nicholas Schiefer Amanda Askell ... Alex Tamkin Janel Thamkul Jared Kaplan Jack Clark Deep Ganguli 35 207 0 28 Jun 2023
Leveraging GPT-4 for Food Effect Summarization to Enhance Product-Specific Guidance Development via Iterative Prompting Yiwen Shi Ping Ren Jing Wang Biao Han Taha ValizadehAslani Felix Agbavor Yi Zhang Meng Hu Liang Zhao Hualou Liang 25 17 0 28 Jun 2023
H $_2$ O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models Zhenyu (Allen) Zhang Ying Sheng Dinesh Manocha Tianlong Chen Lianmin Zheng ... Yuandong Tian Christopher Ré Clark W. Barrett Zhangyang Wang Beidi Chen VLM 52 254 0 24 Jun 2023
System-Level Natural Language Feedback Weizhe Yuan Kyunghyun Cho Jason Weston 38 5 0 23 Jun 2023
Opportunities and Risks of LLMs for Scalable Deliberation with Polis Christopher T. Small Ivan Vendrov Esin Durmus Hadjar Homaei Elizabeth Barry Julien Cornebise Ted Suzman Deep Ganguli Colin Megill 29 26 0 20 Jun 2023
Are Large Language Models Really Good Logical Reasoners? A Comprehensive Evaluation and Beyond Fangzhi Xu Qika Lin Jiawei Han Tianzhe Zhao Jun Liu Erik Cambria ELM LRM 38 31 0 16 Jun 2023
How Ready are Pre-trained Abstractive Models and LLMs for Legal Case Judgement Summarization? Aniket Deroy Kripabandhu Ghosh Saptarshi Ghosh ELM AILaw 13 60 0 02 Jun 2023
Hybrid Long Document Summarization using C2F-FAR and ChatGPT: A Practical Study Guang Lu Sylvia B. Larcher Tu-Anh Tran 19 9 0 01 Jun 2023
Generating EDU Extracts for Plan-Guided Summary Re-Ranking Griffin Adams Alexander R. Fabbri Faisal Ladhak Kathleen McKeown Noémie Elhadad 18 10 0 28 May 2023
MeetingBank: A Benchmark Dataset for Meeting Summarization Yebowen Hu Timothy Jeewun Ganter Hanieh Deilamsalehy Franck Dernoncourt H. Foroosh Fei Liu AI4TS 18 45 0 27 May 2023
Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying References Tianyi Tang Hongyuan Lu Yuchen Eleanor Jiang Haoyang Huang Dongdong Zhang Wayne Xin Zhao Tom Kocmi Furu Wei 20 5 0 24 May 2023
SummIt: Iterative Text Summarization via ChatGPT Haopeng Zhang Xiao Liu Jiawei Zhang 40 64 0 24 May 2023
Using Natural Language Explanations to Rescale Human Judgments Manya Wadhwa Jifan Chen Junyi Jessy Li Greg Durrett 38 8 0 24 May 2023
Trusting Your Evidence: Hallucinate Less with Context-aware Decoding Weijia Shi Xiaochuang Han M. Lewis Yulia Tsvetkov Luke Zettlemoyer Scott Yih HILM 19 189 0 24 May 2023
DecipherPref: Analyzing Influential Factors in Human Preference Judgments via GPT-4 Ye Hu Kaiqiang Song Sangwoo Cho Xiaoyang Wang H. Foroosh Fei Liu 23 11 0 24 May 2023
Sources of Hallucination by Large Language Models on Inference Tasks Nick McKenna Tianyi Li Liang Cheng Mohammad Javad Hosseini Mark Johnson Mark Steedman LRM HILM 16 188 0 23 May 2023
On Learning to Summarize with Large Language Models as References Yixin Liu Kejian Shi Katherine S He Longtian Ye Alexander R. Fabbri Pengfei Liu Dragomir R. Radev Arman Cohan ELM 28 71 0 23 May 2023
Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models Alfonso Amayuelas Kyle Wong Liangming Pan Wenhu Chen Luu Anh Tuan 42 26 0 23 May 2023
Element-aware Summarization with Large Language Models: Expert-aligned Evaluation and Chain-of-Thought Method Yiming Wang Zhuosheng Zhang Rui Wang 44 78 0 22 May 2023
Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline Zangwei Zheng Xiaozhe Ren Fuzhao Xue Yang Luo Xin Jiang Yang You 42 54 0 22 May 2023
Complex Claim Verification with Evidence Retrieved in the Wild Jifan Chen Grace Kim Aniruddh Sriram Greg Durrett Eunsol Choi HILM 22 68 0 19 May 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 39 82 0 19 May 2023
PMIndiaSum: Multilingual and Cross-lingual Headline Summarization for Languages in India Ashok Urlana Pinzhen Chen Zheng Zhao Shay B. Cohen Manish Shrivastava Barry Haddow 29 9 0 15 May 2023
Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical Study Yaxin Fan Feng Jiang Peifeng Li Haizhou Li ELM 32 19 0 15 May 2023
The Current State of Summarization Fabian Retkowski 23 6 0 08 May 2023
Evaluation of GPT-3.5 and GPT-4 for supporting real-world information needs in healthcare delivery Debadutta Dash Rahul Thapa Juan M. Banda Akshay Swaminathan Morgan Cheatham ... Garret K. Morris H. Magon M. Lungren Eric Horvitz N. Shah ELM LM&MA AI4MH 68 51 0 26 Apr 2023
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond Jingfeng Yang Hongye Jin Ruixiang Tang Xiaotian Han Qizhang Feng Haoming Jiang Bing Yin Xia Hu LM&MA 134 622 0 26 Apr 2023
Extractive Summarization via ChatGPT for Faithful Summary Generation Haopeng Zhang Xiao Liu Jiawei Zhang 38 76 0 09 Apr 2023
Beyond Summarization: Designing AI Support for Real-World Expository Writing Tasks Zejiang Shen Tal August Pao Siangliulue Kyle Lo Jonathan Bragg Jeff Hammerbacher Doug Downey Joseph Chee Chang David Sontag ELM 20 18 0 05 Apr 2023
Human-like Summarization Evaluation with ChatGPT Mingqi Gao Jie Ruan Renliang Sun Xunjian Yin Shiping Yang Xiaojun Wan ALM AI4MH 21 125 0 05 Apr 2023
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment Yang Liu Dan Iter Yichong Xu Shuohang Wang Ruochen Xu Chenguang Zhu ELM ALM LM&MA 53 1,078 0 29 Mar 2023
SmartBook: AI-Assisted Situation Report Generation for Intelligence Analysts R. Reddy Daniel Lee Yi Ren Fung Khanh Duy Nguyen Qi Zeng Manling Li Ziqi Wang Clare R. Voss Heng Ji 20 6 0 25 Mar 2023
Who's Thinking? A Push for Human-Centered Evaluation of LLMs using the XAI Playbook Teresa Datta John P. Dickerson 34 10 0 10 Mar 2023
Guiding Large Language Models via Directional Stimulus Prompting Zekun Li Baolin Peng Pengcheng He Michel Galley Jianfeng Gao Xi Yan LLMAG LRM LM&Ro 40 94 0 22 Feb 2023
Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization Xianjun Yang Yan Li Xinlu Zhang Haifeng Chen Wei Cheng AI4MH 31 173 0 16 Feb 2023
Envisioning the Next-Gen Document Reader Catherine Yeh Nedim Lipka Franck Dernoncourt SyDa 17 0 0 15 Feb 2023
SumREN: Summarizing Reported Speech about Events in News R. Reddy Heba Elfardy Hou Pong Chan Kevin Small Heng Ji 24 5 0 02 Dec 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 319 11,953 0 04 Mar 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,657 0 15 Oct 2021
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 280 1,595 0 18 Sep 2019
Text Summarization with Pretrained Encoders Yang Liu Mirella Lapata MILM 258 1,432 0 22 Aug 2019
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 175 3,510 0 10 Jun 2015