Can Transformer Models Measure Coherence In Text? Re-Thinking the
Shuffle Test

Can Transformer Models Measure Coherence In Text? Re-Thinking the Shuffle Test

7 July 2021

Lucas Bandarkar

Marti A. Hearst

Papers citing "Can Transformer Models Measure Coherence In Text? Re-Thinking the Shuffle Test"

15 / 15 papers shown

Title
Unlocking Structure Measuring: Introducing PDD, an Automatic Metric for Positional Discourse Coherence Yinhong Liu Yixuan Su Ehsan Shareghi Nigel Collier 36 4 0 15 Feb 2024
BBScore: A Brownian Bridge Based Metric for Assessing Text Coherence Zhecheng Sheng Tianhao Zhang Chen Jiang Dongyeop Kang 29 5 0 28 Dec 2023
HelpSteer: Multi-attribute Helpfulness Dataset for SteerLM Zhilin Wang Yi Dong Jiaqi Zeng Virginia Adams Makesh Narsimhan Sreedhar ... Olivier Delalleau Jane Polak Scowcroft Neel Kant Aidan Swope Oleksii Kuchaiev 3DV 22 67 0 16 Nov 2023
A Digital Language Coherence Marker for Monitoring Dementia Dimitris Gkoumas Adam Tsakalidis M. Liakata 16 1 0 14 Oct 2023
A Novel Computational and Modeling Foundation for Automatic Coherence Assessment Aviya Maimon Reut Tsarfaty 82 5 0 01 Oct 2023
Enhancing Coherence of Extractive Summarization with Multitask Learning Renlong Jie Xiaojun Meng Lifeng Shang Xin Jiang Qun Liu 25 1 0 22 May 2023
In-context Learning as Maintaining Coherency: A Study of On-the-fly Machine Translation Using Large Language Models Suzanna Sia Kevin Duh LRM 23 17 0 05 May 2023
Combat AI With AI: Counteract Machine-Generated Fake Restaurant Reviews on Social Media Alessandro Gambetti Qiwei Han DeLMO 21 9 0 10 Feb 2023
How to Find Strong Summary Coherence Measures? A Toolbox and a Comparative Study for Summary Coherence Measure Evaluation Julius Steen K. Markert HILM 19 4 0 14 Sep 2022
GisPy: A Tool for Measuring Gist Inference Score in Text Pedram Hosseini C. Wolfe Mona T. Diab David A. Broniatowski 22 2 0 25 May 2022
DiscoScore: Evaluating Text Generation with BERT and Discourse Coherence Wei Zhao Michael Strube Steffen Eger 27 37 0 26 Jan 2022
An Application of Pseudo-Log-Likelihoods to Natural Language Scoring Darren Abramson Ali Emami 46 3 0 23 Jan 2022
Imagined versus Remembered Stories: Quantifying Differences in Narrative Flow Maarten Sap A. Jafarpour Yejin Choi Noah A. Smith J. Pennebaker Eric Horvitz 16 1 0 07 Jan 2022
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little Koustuv Sinha Robin Jia Dieuwke Hupkes J. Pineau Adina Williams Douwe Kiela 45 244 0 14 Apr 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 288 2,028 0 28 Jul 2020