Measuring and Improving Consistency in Pretrained Language Models

1 February 2021

Yanai Elazar

Nora Kassner

Shauli Ravfogel

Abhilasha Ravichander

Papers citing "Measuring and Improving Consistency in Pretrained Language Models"

50 / 60 papers shown

Title
Consistency in Language Models: Current Landscape, Challenges, and Future Directions Jekaterina Novikova Carol Anderson Borhane Blili-Hamelin Subhabrata Majumdar HILM 69 0 0 01 May 2025
MAC-Tuning: LLM Multi-Compositional Problem Reasoning with Enhanced Knowledge Boundary Awareness Junsheng Huang Zhitao He Sandeep Polisetty Q. Wang May Fung KELM 45 0 0 30 Apr 2025
Generative AI for Research Data Processing: Lessons Learnt From Three Use Cases Modhurita Mitra Martine G. de Vos Nicola Cortinovis Dawa Ometto 22 0 0 22 Apr 2025
FACT-AUDIT: An Adaptive Multi-Agent Framework for Dynamic Fact-Checking Evaluation of Large Language Models Hongzhan Lin Yang Deng Yuxuan Gu Wenxuan Zhang Jing Ma See-Kiong Ng Tat-Seng Chua LLMAG KELM HILM 65 0 0 25 Feb 2025
The Knowledge Microscope: Features as Better Analytical Lenses than Neurons Yuheng Chen Pengfei Cao Kang Liu Jun Zhao 43 0 0 18 Feb 2025
Enhancing Semantic Consistency of Large Language Models through Model Editing: An Interpretability-Oriented Approach J. Yang Dapeng Chen Yajing Sun Rongjun Li Zhiyong Feng Wei Peng 51 5 0 19 Jan 2025
On the Consistency of Video Large Language Models in Temporal Comprehension Minjoon Jung Junbin Xiao Byoung-Tak Zhang Angela Yao 87 2 0 20 Nov 2024
Fact Recall, Heuristics or Pure Guesswork? Precise Interpretations of Language Models for Fact Completion Denitsa Saynova Lovisa Hagström Moa Johansson Richard Johansson Marco Kuhlmann HILM 41 0 0 18 Oct 2024
One Language, Many Gaps: Evaluating Dialect Fairness and Robustness of Large Language Models in Reasoning Tasks Fangru Lin Shaoguang Mao Emanuele La Malfa Valentin Hofmann Adrian de Wynter Jing Yao Si-Qing Chen Michael Wooldridge Furu Wei Furu Wei 51 2 0 14 Oct 2024
Understanding the Interplay between Parametric and Contextual Knowledge for Large Language Models Sitao Cheng Liangming Pan Xunjian Yin Xinyi Wang William Yang Wang KELM 37 4 0 10 Oct 2024
MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models Shengkang Wang Hongzhan Lin Ziyang Luo Zhen Ye Guang Chen Jing Ma 65 3 0 17 Jun 2024
Outdated Issue Aware Decoding for Reasoning Questions on Edited Knowledge Zengkui Sun Yijin Liu Jiaan Wang Fandong Meng Jinan Xu Yufeng Chen Jie Zhou KELM 35 0 0 05 Jun 2024
Hire Me or Not? Examining Language Model's Behavior with Occupation Attributes Damin Zhang Yi Zhang Geetanjali Bihani Julia Taylor Rayz 50 2 0 06 May 2024
What does the Knowledge Neuron Thesis Have to do with Knowledge? Jingcheng Niu Andrew Liu Zining Zhu Gerald Penn 48 30 0 03 May 2024
A Framework for Real-time Safeguarding the Text Generation of Large Language Model Ximing Dong Dayi Lin Shaowei Wang Ahmed E. Hassan 33 1 0 29 Apr 2024
Navigating the Path of Writing: Outline-guided Text Generation with Large Language Models Yukyung Lee Soonwon Ka Bokyung Son Pilsung Kang Jaewook Kang LLMAG 49 6 0 22 Apr 2024
SemEval-2024 Task 2: Safe Biomedical Natural Language Inference for Clinical Trials Mael Jullien Marco Valentino André Freitas LM&MA 36 41 0 07 Apr 2024
"Flex Tape Can't Fix That": Bias and Misinformation in Edited Language Models Karina Halevy Anna Sotnikova Badr AlKhamissi Syrielle Montariol Antoine Bosselut KELM 34 3 0 29 Feb 2024
On the Challenges and Opportunities in Generative AI Laura Manduchi Kushagra Pandey Robert Bamler Ryan Cotterell Sina Daubener ... F. Wenzel Frank Wood Stephan Mandt Vincent Fortuin Vincent Fortuin 56 17 0 28 Feb 2024
SaGE: Evaluating Moral Consistency in Large Language Models Vamshi Bonagiri Sreeram Vennam Priyanshul Govil Ponnurangam Kumaraguru Manas Gaur ELM 54 0 0 21 Feb 2024
The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction Pratyusha Sharma Jordan T. Ash Dipendra Kumar Misra LRM 15 78 0 21 Dec 2023
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia Giovanni Monea Maxime Peyrard Martin Josifoski Vishrav Chaudhary Jason Eisner Emre Kiciman Hamid Palangi Barun Patra Robert West KELM 51 12 0 04 Dec 2023
R-Tuning: Instructing Large Language Models to Say `I Don't Know' Hanning Zhang Shizhe Diao Yong Lin Yi Ren Fung Qing Lian Xingyao Wang Yangyi Chen Heng Ji Tong Zhang UQLM 36 36 0 16 Nov 2023
Assessing Knowledge Editing in Language Models via Relation Perspective Yifan Wei Xiaoyan Yu Huanhuan Ma Fangyu Lei Yixuan Weng Ran Song Kang Liu KELM 28 15 0 15 Nov 2023
Multi-Agent Consensus Seeking via Large Language Models Huaben Chen Wenkang Ji Lufeng Xu Shiyu Zhao LM&Ro LLMAG 61 20 0 31 Oct 2023
Who Are All The Stochastic Parrots Imitating? They Should Tell Us! Sagi Shaier Lawrence E Hunter K. Wense 34 3 0 16 Oct 2023
Configuration Validation with Large Language Models Xinyu Lian Yinfang Chen Runxiang Cheng Jie Huang Parth Thakkar Minjia Zhang Tianyin Xu 13 10 0 15 Oct 2023
Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models Haoran Wang Kai Shu LRM 43 22 0 08 Oct 2023
Semantic Consistency for Assuring Reliability of Large Language Models Harsh Raj Vipul Gupta Domenic Rosati S. Majumdar HILM 102 14 0 17 Aug 2023
XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models Paul Röttger Hannah Rose Kirk Bertie Vidgen Giuseppe Attanasio Federico Bianchi Dirk Hovy ALM ELM AILaw 23 122 0 02 Aug 2023
Personality Traits in Large Language Models Gregory Serapio-García Mustafa Safdari Clément Crepy Luning Sun Stephen Fitz P. Romero Marwa Abdulhai Aleksandra Faust Maja J. Matarić LM&MA LLMAG 58 119 0 01 Jul 2023
Prompt to be Consistent is Better than Self-Consistent? Few-Shot and Zero-Shot Fact Verification with Pre-trained Language Models Fengzhu Zeng Wei Gao 17 5 0 05 Jun 2023
Flatness-Aware Prompt Selection Improves Accuracy and Sample Efficiency Lingfeng Shen Weiting Tan Boyuan Zheng Daniel Khashabi VLM 39 6 0 18 May 2023
Statistical Knowledge Assessment for Large Language Models Qingxiu Dong Jingjing Xu Lingpeng Kong Zhifang Sui Lei Li HILM 33 6 0 17 May 2023
Psychologically-Inspired Causal Prompts Zhiheng Lyu Zhijing Jin Justus Mattern Rada Mihalcea Mrinmaya Sachan Bernhard Schoelkopf CML 31 0 0 02 May 2023
Progressive-Hint Prompting Improves Reasoning in Large Language Models Chuanyang Zheng Zhengying Liu Enze Xie Zhenguo Li Yu Li LLMAG ReLM LRM 26 101 0 19 Apr 2023
Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open World Qifan Yu Juncheng Li Yuehua Wu Siliang Tang Wei Ji Yueting Zhuang 25 34 0 23 Mar 2023
Dissociating language and thought in large language models Kyle Mahowald Anna A. Ivanova I. Blank Nancy Kanwisher J. Tenenbaum Evelina Fedorenko ELM ReLM 25 209 0 16 Jan 2023
Event knowledge in large language models: the gap between the impossible and the unlikely Carina Kauf Anna A. Ivanova Giulia Rambelli Emmanuele Chersoni Jingyuan Selena She Zawad Chowdhury Evelina Fedorenko Alessandro Lenci 37 67 0 02 Dec 2022
Measuring Reliability of Large Language Models through Semantic Consistency Harsh Raj Domenic Rosati S. Majumdar HILM 22 30 0 10 Nov 2022
The Effect of Normalization for Bi-directional Amharic-English Neural Machine Translation Tadesse Destaw Belay A. Tonja Olga Kolesnikova Seid Muhie Yimam A. Ayele Sileshi Bogale Haile Grigori Sidorov Alexander Gelbukh CVBM 27 5 0 27 Oct 2022
Pre-training Language Models with Deterministic Factual Knowledge Shaobo Li Xiaoguang Li Lifeng Shang Chengjie Sun Bingquan Liu Zhenzhou Ji Xin Jiang Qun Liu KELM 36 11 0 20 Oct 2022
Query Expansion Using Contextual Clue Sampling with Language Models Linqing Liu Minghan Li Jimmy J. Lin Sebastian Riedel Pontus Stenetorp 32 10 0 13 Oct 2022
Understanding Transformer Memorization Recall Through Idioms Adi Haviv Ido Cohen Jacob Gidron R. Schuster Yoav Goldberg Mor Geva 26 48 0 07 Oct 2022
Calibrating Factual Knowledge in Pretrained Language Models Qingxiu Dong Damai Dai Yifan Song Jingjing Xu Zhifang Sui Lei Li KELM 230 82 0 07 Oct 2022
Lost in Context? On the Sense-wise Variance of Contextualized Word Embeddings Yile Wang Yue Zhang 17 4 0 20 Aug 2022
Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations Jaehun Jung Lianhui Qin Sean Welleck Faeze Brahman Chandra Bhagavatula Ronan Le Bras Yejin Choi ReLM LRM 223 190 0 24 May 2022
TempLM: Distilling Language Models into Template-Based Generators Tianyi Zhang Mina Lee Lisa Li Ende Shen Tatsunori B. Hashimoto VLM 32 5 0 23 May 2022
Language Models in the Loop: Incorporating Prompting into Weak Supervision Ryan Smith Jason Alan Fries Braden Hancock Stephen H. Bach 47 53 0 04 May 2022
Prompt Consistency for Zero-Shot Task Generalization Chunting Zhou Junxian He Xuezhe Ma Taylor Berg-Kirkpatrick Graham Neubig VLM 8 74 0 29 Apr 2022