Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly

8 November 2019

Papers citing "Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly"

50 / 211 papers shown

Title
Assessing Step-by-Step Reasoning against Lexical Negation: A Case Study on Syllogism Mengyu Ye Tatsuki Kuribayashi Jun Suzuki Goro Kobayashi Hiroaki Funayama LRM 34 8 0 23 Oct 2023
Emptying the Ocean with a Spoon: Should We Edit Models? Yuval Pinter Michael Elhadad KELM 27 26 0 18 Oct 2023
Revealing the Unwritten: Visual Investigation of Beam Search Trees to Address Language Model Prompting Challenges Thilo Spinner Rebecca Kehlbeck Rita Sevastjanova Tobias Stähle Daniel A. Keim Oliver Deussen Andreas Spitz Mennatallah El-Assady 32 2 0 17 Oct 2023
Assessing the Reliability of Large Language Model Knowledge Weixuan Wang Barry Haddow Alexandra Birch Wei Peng KELM HILM 80 15 0 15 Oct 2023
Impact of Co-occurrence on Factual Knowledge of Large Language Models Cheongwoong Kang Jaesik Choi KELM 38 17 0 12 Oct 2023
Evaluating Large Language Models at Evaluating Instruction Following Zhiyuan Zeng Jiatong Yu Tianyu Gao Yu Meng Tanya Goyal Danqi Chen ELM ALM 49 170 0 11 Oct 2023
Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models Holy Lovenia Wenliang Dai Samuel Cahyawijaya Ziwei Ji Pascale Fung MLLM 36 51 0 09 Oct 2023
Making Retrieval-Augmented Language Models Robust to Irrelevant Context Ori Yoran Tomer Wolfson Ori Ram Jonathan Berant RALM LRM 26 187 0 02 Oct 2023
RelBERT: Embedding Relations with Language Models Asahi Ushio Jose Camacho-Collados Steven Schockaert KELM 22 1 0 30 Sep 2023
Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayings Chen Cecilia Liu Fajri Koto Timothy Baldwin Iryna Gurevych LRM 32 18 0 15 Sep 2023
In-Contextual Gender Bias Suppression for Large Language Models Daisuke Oba Masahiro Kaneko Danushka Bollegala 31 8 0 13 Sep 2023
Characterizing Latent Perspectives of Media Houses Towards Public Figures S. Srivatsa Srinath Srinivasa 20 0 0 12 Sep 2023
Do PLMs Know and Understand Ontological Knowledge? Weiqi Wu Chengyue Jiang Yong-jia Jiang Pengjun Xie Kewei Tu 28 25 0 12 Sep 2023
Large Language Models and Knowledge Graphs: Opportunities and Challenges Jeff Z. Pan Simon Razniewski Jan-Christoph Kalo Sneha Singhania Jiaoyan Chen ... Gerard de Melo A. Bonifati Edlira Vakaj M. Dragoni D. Graux KELM 30 73 0 11 Aug 2023
Learning Deductive Reasoning from Synthetic Corpus based on Formal Logic Terufumi Morishita Gaku Morio Atsuki Yamaguchi Yasuhiro Sogawa ReLM LRM AI4CE ELM 38 23 0 11 Aug 2023
This is not correct! Negation-aware Evaluation of Language Generation Systems Miriam Anschütz Diego Miguel Lozano Georg Groh 38 6 0 26 Jul 2023
Thrust: Adaptively Propels Large Language Models with External Knowledge Xinran Zhao Hongming Zhang Xiaoman Pan Wenlin Yao Dong Yu Jianshu Chen KELM 60 5 0 19 Jul 2023
Knowledge Base Completion for Long-Tail Entities Lihu Chen Simon Razniewski Gerhard Weikum KELM 26 6 0 30 Jun 2023
Limits for Learning with Language Models Nicholas M. Asher Swarnadeep Bhar Akshay Chaturvedi Julie Hunter Soumya Paul 27 22 0 21 Jun 2023
Towards Understanding What Code Language Models Learned Toufique Ahmed Dian Yu Chen Huang Cathy Wang Prem Devanbu Kenji Sagae ELM 46 5 0 20 Jun 2023
CMMLU: Measuring massive multitask language understanding in Chinese Haonan Li Yixuan Zhang Fajri Koto Yifei Yang Hai Zhao Yeyun Gong Nan Duan Tim Baldwin ALM ELM 47 245 0 15 Jun 2023
Language models are not naysayers: An analysis of language models on negation benchmarks Thinh Hung Truong Timothy Baldwin Karin Verspoor Trevor Cohn 40 54 0 14 Jun 2023
Probing Quantifier Comprehension in Large Language Models: Another Example of Inverse Scaling Akshat Gupta ELM LRM 26 7 0 12 Jun 2023
Prompt to be Consistent is Better than Self-Consistent? Few-Shot and Zero-Shot Fact Verification with Pre-trained Language Models Fengzhu Zeng Wei Gao 23 5 0 05 Jun 2023
SLABERT Talk Pretty One Day: Modeling Second Language Acquisition with BERT Aditya Yadavalli Alekhya Yadavalli Vera Tobin 11 6 0 31 May 2023
ScoNe: Benchmarking Negation Reasoning in Language Models With Fine-Tuning and In-Context Learning Jingyuan Selena She Christopher Potts Sam Bowman Atticus Geiger 32 13 0 30 May 2023
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models Yuhui Zhang Michihiro Yasunaga Zhengping Zhou Jeff Z. HaoChen James Zou Percy Liang Serena Yeung 47 7 0 27 May 2023
Calibration of Transformer-based Models for Identifying Stress and Depression in Social Media Loukas Ilias S. Mouzakitis D. Askounis 33 41 0 26 May 2023
Can large language models generate salient negative statements? Hiba Arnaout Simon Razniewski 20 3 0 26 May 2023
Simple Linguistic Inferences of Large Language Models (LLMs): Blind Spots and Blinds Victoria Basmov Yoav Goldberg Reut Tsarfaty ReLM LRM 32 5 0 24 May 2023
Testing Causal Models of Word Meaning in GPT-3 and -4 Sam Musker Ellie Pavlick 18 1 0 24 May 2023
Language Models with Rationality Nora Kassner Oyvind Tafjord Ashish Sabharwal Kyle Richardson Hinrich Schütze Peter Clark ReLM KELM LRM 20 15 0 23 May 2023
HumBEL: A Human-in-the-Loop Approach for Evaluating Demographic Factors of Language Models in Human-Machine Conversations Anthony Sicilia Jennifer C. Gates Malihe Alikhani 11 4 0 23 May 2023
Assessing Linguistic Generalisation in Language Models: A Dataset for Brazilian Portuguese Rodrigo Wilkens Leonardo Zilio Aline Villavicencio 24 1 0 23 May 2023
Separating form and meaning: Using self-consistency to quantify task understanding across multiple senses Xenia Ohmer Elia Bruni Dieuwke Hupkes LRM 29 13 0 19 May 2023
Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledge Jiangjie Chen Wei Shi Ziquan Fu Sijie Cheng Lei Li Yanghua Xiao 35 46 0 10 May 2023
ANALOGICAL -- A Novel Benchmark for Long Text Analogy Evaluation in Large Language Models Thilini Wijesiriwardene Ruwan Wickramarachchi Bimal Gajera Shreeyash Mukul Gowaikar Chandan Gupta Aman Chadha Aishwarya N. Reganti Amit P. Sheth Amitava Das ELM 25 14 0 08 May 2023
Evaluating statistical language models as pragmatic reasoners Benjamin Lipkin L. Wong Gabriel Grand J. Tenenbaum 49 14 0 01 May 2023
Language Model Behavior: A Comprehensive Survey Tyler A. Chang Benjamin Bergen VLM LRM LM&MA 29 103 0 20 Mar 2023
Large Language Model Instruction Following: A Survey of Progresses and Challenges Renze Lou Kai Zhang Wenpeng Yin ALM LRM 35 20 0 18 Mar 2023
The Life Cycle of Knowledge in Big Language Models: A Survey Boxi Cao Hongyu Lin Xianpei Han Le Sun KELM 33 27 0 14 Mar 2023
Consistency Analysis of ChatGPT Myeongjun Jang Thomas Lukasiewicz 30 53 0 11 Mar 2023
The Contribution of Knowledge in Visiolinguistic Learning: A Survey on Tasks and Challenges Maria Lymperaiou Giorgos Stamou VLM 32 4 0 04 Mar 2023
Competence-Based Analysis of Language Models Adam Davies Jize Jiang Chengxiang Zhai ELM 29 4 0 01 Mar 2023
In-Depth Look at Word Filling Societal Bias Measures Matúš Pikuliak Ivana Benová Viktor Bachratý 29 9 0 24 Feb 2023
Dynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views Katerina Margatina Shuai Wang Yogarshi Vyas Neha Ann John Yassine Benajiba Miguel Ballesteros 17 15 0 23 Feb 2023
FiTs: Fine-grained Two-stage Training for Knowledge-aware Question Answering Qichen Ye Bowen Cao Nuo Chen Weiyuan Xu Yuexian Zou 26 16 0 23 Feb 2023
Large Language Models Can Be Easily Distracted by Irrelevant Context Freda Shi Xinyun Chen Kanishka Misra Nathan Scales David Dohan Ed H. Chi Nathanael Scharli Denny Zhou ReLM RALM LRM 35 541 0 31 Jan 2023
Can Peanuts Fall in Love with Distributional Semantics? J. Michaelov S. Coulson Benjamin Bergen MILM 26 8 0 20 Jan 2023
Dissociating language and thought in large language models Kyle Mahowald Anna A. Ivanova I. Blank Nancy Kanwisher J. Tenenbaum Evelina Fedorenko ELM ReLM 31 209 0 16 Jan 2023