Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models

22 May 2025

Papers citing "Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models"

33 / 33 papers shown

Title
Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Models Mingyang Wang Heike Adel Lukas Lange Yihong Liu Ercong Nie Jannik Strötgen Hinrich Schütze HILM 77 5 0 05 Apr 2025
Interpreting Arithmetic Mechanism in Large Language Models through Comparative Neuron Analysis Zeping Yu Sophia Ananiadou LRM MILM 68 11 0 21 Sep 2024
Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in? Chengzhi Zhong Fei Cheng Qianying Liu Junfeng Jiang Zhen Wan Chenhui Chu Yugo Murawaki Sadao Kurohashi LRM 54 19 0 20 Aug 2024
Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons Yifei Wang Yuheng Chen Wanting Wen Yu Sheng Linjing Li D. Zeng KELM 66 8 0 06 Aug 2024
Understanding and Mitigating Language Confusion in LLMs Kelly Marchisio Wei-Yin Ko Alexandre Berard Théo Dehaze Sebastian Ruder 87 27 0 28 Jun 2024
Unlocking the Future: Exploring Look-Ahead Planning Mechanistic Interpretability in Large Language Models Tianyi Men Pengfei Cao Zhuoran Jin Yubo Chen Kang Liu Jun Zhao LLMAG AIFin 56 7 0 23 Jun 2024
Key ingredients for effective zero-shot cross-lingual knowledge transfer in generative tasks Nadezhda Chirkova Vassilina Nikoulina 41 9 0 19 Feb 2024
Do Llamas Work in English? On the Latent Language of Multilingual Transformers Chris Wendler V. Veselovsky Giovanni Monea Robert West 92 119 0 16 Feb 2024
Multilingual Instruction Tuning With Just a Pinch of Multilinguality Uri Shaham Jonathan Herzig Roee Aharoni Idan Szpektor Reut Tsarfaty Matan Eyal LRM 51 46 0 03 Jan 2024
LLaMA Beyond English: An Empirical Study on Language Capability Transfer Jun Zhao Zhihao Zhang Luhui Gao Qi Zhang Tao Gui Xuanjing Huang ELM 45 73 0 02 Jan 2024
Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed? Tannon Kew Florian Schottmann Rico Sennrich LRM 57 37 0 20 Dec 2023
Future Lens: Anticipating Subsequent Tokens from a Single Hidden State Koyena Pal Jiuding Sun Andrew Yuan Byron C. Wallace David Bau 40 55 0 08 Nov 2023
Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decoding Rico Sennrich Jannis Vamvas Alireza Mohammadshahi HILM 104 41 0 13 Sep 2023
Extrapolating Large Language Models to Non-English by Aligning Languages Wenhao Zhu Yunzhe Lv Qingxiu Dong Fei Yuan Jingjing Xu Shujian Huang Lingpeng Kong Jiajun Chen Lei Li 58 67 0 09 Aug 2023
Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback Viet Dac Lai Chien Van Nguyen Nghia Trung Ngo Thuat Nguyen Franck Dernoncourt Ryan Rossi Thien Huu Nguyen ALM 64 139 0 29 Jul 2023
Why Does Zero-Shot Cross-Lingual Generation Fail? An Explanation and a Solution Tianjian Li Kenton W. Murray 66 25 0 27 May 2023
BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer Akari Asai Sneha Kudugunta Xinyan Velocity Yu Terra Blevins Hila Gonen Machel Reid Yulia Tsvetkov Sebastian Ruder Hannaneh Hajishirzi 79 60 0 24 May 2023
mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations Jonas Pfeiffer Francesco Piccinno Massimo Nicosia Xinyi Wang Machel Reid Sebastian Ruder VLM LRM 58 29 0 23 May 2023
Towards Automated Circuit Discovery for Mechanistic Interpretability Arthur Conmy Augustine N. Mavor-Parker Aengus Lynch Stefan Heimersheim Adrià Garriga-Alonso 45 301 0 28 Apr 2023
Dissecting Recall of Factual Associations in Auto-Regressive Language Models Mor Geva Jasmijn Bastings Katja Filippova Amir Globerson KELM 228 297 0 28 Apr 2023
MEGA: Multilingual Evaluation of Generative AI Kabir Ahuja Harshita Diddee Rishav Hada Millicent Ochieng Krithika Ramesh ... T. Ganu Sameer Segal Maxamed Axmed Kalika Bali Sunayana Sitaram LM&MA LRM ELM 47 282 0 22 Mar 2023
Eliciting Latent Predictions from Transformers with the Tuned Lens Nora Belrose Zach Furman Logan Smith Danny Halawi Igor V. Ostrovsky Lev McKinney Stella Biderman Jacob Steinhardt 38 213 0 14 Mar 2023
A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies A. Seza Doğruöz Sunayana Sitaram Barbara E. Bullock Almeida Jacqueline Toribio 88 76 0 05 Jan 2023
The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges Genta Indra Winata Alham Fikri Aji Zheng-Xin Yong Thamar Solorio 69 36 0 19 Dec 2022
Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small Kevin Wang Alexandre Variengien Arthur Conmy Buck Shlegeris Jacob Steinhardt 275 531 0 01 Nov 2022
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space Mor Geva Avi Caciularu Ke Wang Yoav Goldberg KELM 83 358 0 28 Mar 2022
Locating and Editing Factual Associations in GPT Kevin Meng David Bau A. Andonian Yonatan Belinkov KELM 154 1,308 0 10 Feb 2022
Transformer Feed-Forward Layers Are Key-Value Memories Mor Geva R. Schuster Jonathan Berant Omer Levy KELM 111 792 0 29 Dec 2020
Understanding the Role of Individual Units in a Deep Neural Network David Bau Jun-Yan Zhu Hendrik Strobelt Àgata Lapedriza Bolei Zhou Antonio Torralba GAN 42 446 0 10 Sep 2020
GLUECoS : An Evaluation Benchmark for Code-Switched NLP Simran Khanuja Sandipan Dandapat A. Srinivasan Sunayana Sitaram Monojit Choudhury ELM 41 147 0 26 Apr 2020
XNLI: Evaluating Cross-lingual Sentence Representations Alexis Conneau Guillaume Lample Ruty Rinott Adina Williams Samuel R. Bowman Holger Schwenk Veselin Stoyanov ELM 53 1,366 0 13 Sep 2018
FastText.zip: Compressing text classification models Armand Joulin Edouard Grave Piotr Bojanowski Matthijs Douze Hervé Jégou Tomas Mikolov MQ 67 1,201 0 12 Dec 2016
Bag of Tricks for Efficient Text Classification Armand Joulin Edouard Grave Piotr Bojanowski Tomas Mikolov VLM 97 4,596 0 06 Jul 2016