Separating form and meaning: Using self-consistency to quantify task
understanding across multiple senses

Separating form and meaning: Using self-consistency to quantify task understanding across multiple senses

19 May 2023

Papers citing "Separating form and meaning: Using self-consistency to quantify task understanding across multiple senses"

16 / 16 papers shown

Title
MultiLoKo: a multilingual local knowledge benchmark for LLMs spanning 31 languages Dieuwke Hupkes Nikolay Bogoychev 142 0 0 14 Apr 2025
ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfer Omer Goldman Uri Shaham Dan Malkin Sivan Eiger Avinatan Hassidim ... Shruti Rijhwani Laura Rimell Idan Szpektor Reut Tsarfaty Matan Eyal 47 3 0 28 Feb 2025
Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models Lynn Chua Badih Ghazi Yangsibo Huang Pritish Kamath Ravi Kumar Pasin Manurangsi Amer Sinha Chulin Xie Chiyuan Zhang 66 1 0 23 Jun 2024
Factual Confidence of LLMs: on Reliability and Robustness of Current Estimators Matéo Mahaut Laura Aina Paula Czarnowska Momchil Hardalov Thomas Müller Lluís Marquez HILM 40 12 0 19 Jun 2024
MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMs Vera Neplenbroek Arianna Bisazza Raquel Fernández 39 6 0 11 Jun 2024
From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency Xenia Ohmer Elia Bruni Dieuwke Hupkes AI4CE 36 6 0 18 Apr 2024
Knowledge Conflicts for LLMs: A Survey Rongwu Xu Zehan Qi Zhijiang Guo Cunxiang Wang Hongru Wang Yue Zhang Wei Xu 200 94 0 13 Mar 2024
Exploring Multilingual Concepts of Human Value in Large Language Models: Is Value Alignment Consistent, Transferable and Controllable across Languages? Shaoyang Xu Weilong Dong Zishan Guo Xinwei Wu Deyi Xiong 44 6 0 28 Feb 2024
Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain Marcus J. Min Yangruibo Ding Luca Buratti Saurabh Pujar Gail E. Kaiser Suman Jana Baishakhi Ray LRM HILM 27 18 0 21 Oct 2023
Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models Jirui Qi Raquel Fernández Arianna Bisazza KELM HILM 27 60 0 16 Oct 2023
The Debate Over Understanding in AI's Large Language Models Melanie Mitchell D. Krakauer ELM 74 203 0 14 Oct 2022
State-of-the-art generalisation research in NLP: A taxonomy and review Dieuwke Hupkes Mario Giulianelli Verna Dankers Mikel Artetxe Yanai Elazar ... Leila Khalatbari Maria Ryskina Rita Frieske Ryan Cotterell Zhijing Jin 121 94 0 06 Oct 2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models Xuezhi Wang Jason W. Wei Dale Schuurmans Quoc Le Ed H. Chi Sharan Narang Aakanksha Chowdhery Denny Zhou ReLM BDL LRM AI4CE 314 3,273 0 21 Mar 2022
BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief Nora Kassner Oyvind Tafjord Hinrich Schütze Peter Clark KELM LRM 245 64 0 29 Sep 2021
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 269 346 0 01 Feb 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,984 0 20 Apr 2018