Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluation

22 May 2024

Papers citing "Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluation"

29 / 29 papers shown

Title
SCORE: Story Coherence and Retrieval Enhancement for AI Narratives Qiang Yi Yangfan He Jing Wang Xinyuan Song Shiyao Qian ... Kuan Lu Menghao Huo Jiaqi Chen Tianyu Shi Tianyu Shi RALM 95 15 0 30 Mar 2025
Evaluating Creative Short Story Generation in Humans and Large Language Models Mete Ismayilzada Claire Stevenson Lonneke van der Plas LM&MA LRM 88 4 0 04 Nov 2024
Agents' Room: Narrative Generation through Multi-step Collaboration Fantine Huot Reinald Kim Amplayo Jennimaria Palomaki Alice Shoshana Jakobovits Elizabeth Clark Mirella Lapata 87 13 0 03 Oct 2024
Detecting Pretraining Data from Large Language Models Weijia Shi Anirudh Ajith Mengzhou Xia Yangsibo Huang Daogao Liu Terra Blevins Danqi Chen Luke Zettlemoyer MIALM 71 188 0 25 Oct 2023
On the Limitations of Reference-Free Evaluations of Generated Text Daniel Deutsch Rotem Dror Dan Roth 103 46 0 22 Oct 2022
The Glass Ceiling of Automatic Evaluation in Natural Language Generation Pierre Colombo Maxime Peyrard Nathan Noiry Robert West Pablo Piantanida 172 11 0 31 Aug 2022
Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generation Cyril Chhun Pierre Colombo Chloé Clavel Fabian M. Suchanek 119 53 0 24 Aug 2022
Emergent Abilities of Large Language Models Jason W. Wei Yi Tay Rishi Bommasani Colin Raffel Barret Zoph ... Tatsunori Hashimoto Oriol Vinyals Percy Liang J. Dean W. Fedus ELM ReLM LRM 277 2,474 0 15 Jun 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 474 6,231 0 05 Apr 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 874 12,916 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 814 9,351 0 28 Jan 2022
LaMDA: Language Models for Dialog Applications R. Thoppilan Daniel De Freitas Jamie Hall Noam M. Shazeer Apoorv Kulshreshtha ... Blaise Aguera-Arcas Claire Cui M. Croak Ed H. Chi Quoc Le ALM 134 1,593 0 20 Jan 2022
A Temporal Variational Model for Story Generation David Wilmot Frank Keller DRL 79 9 0 14 Sep 2021
Want To Reduce Labeling Cost? GPT-3 Can Help Shuohang Wang Yang Liu Yichong Xu Chenguang Zhu Michael Zeng 66 257 0 30 Aug 2021
Automatic Text Evaluation through the Lens of Wasserstein Barycenters Pierre Colombo Guillaume Staerman Chloé Clavel Pablo Piantanida 155 41 0 27 Aug 2021
All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text Elizabeth Clark Tal August Sofia Serrano Nikita Haduong Suchin Gururangan Noah A. Smith DeLMO 96 410 0 30 Jun 2021
BARTScore: Evaluating Generated Text as Text Generation Weizhe Yuan Graham Neubig Pengfei Liu 99 843 0 22 Jun 2021
Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence Jian Guan Xiaoxi Mao Changjie Fan Zitao Liu Wenbiao Ding Minlie Huang AuLLM 90 81 0 19 May 2021
Automatic Story Generation: Challenges and Attempts Amal Alabdulkarim Siyan Li Xiangyu Peng 68 51 0 25 Feb 2021
Re-evaluating Evaluation in Text Summarization Manik Bhandari Pranav Narayan Gour A. Ashfaq Pengfei Liu Graham Neubig 142 177 0 14 Oct 2020
Evaluation of Text Generation: A Survey Asli Celikyilmaz Elizabeth Clark Jianfeng Gao ELM LM&MA 104 386 0 26 Jun 2020
SUPERT: Towards New Frontiers in Unsupervised Evaluation Metrics for Multi-Document Summarization Yang Gao Wei Zhao Steffen Eger ELM 79 125 0 07 May 2020
PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking Hannah Rashkin Asli Celikyilmaz Yejin Choi Jianfeng Gao 54 153 0 30 Apr 2020
Fill in the BLANC: Human-free quality estimation of document summaries Oleg V. Vasilyev Vedant Dharnidharka John Bohannon 3DH 80 118 0 23 Feb 2020
A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation Jian Guan Fei Huang Zhihao Zhao Xiaoyan Zhu Minlie Huang LRM SyDa 56 247 0 15 Jan 2020
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 1.3K 12,193 0 27 Aug 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 639 24,431 0 26 Jul 2019
XLNet: Generalized Autoregressive Pretraining for Language Understanding Zhilin Yang Zihang Dai Yiming Yang J. Carbonell Ruslan Salakhutdinov Quoc V. Le AI4CE 230 8,426 0 19 Jun 2019
BERTScore: Evaluating Text Generation with BERT Tianyi Zhang Varsha Kishore Felix Wu Kilian Q. Weinberger Yoav Artzi 324 5,801 0 21 Apr 2019