It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning

22 June 2021

Papers citing "It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning"

16 / 16 papers shown

Title
Rewriting Pre-Training Data Boosts LLM Performance in Math and Code Kazuki Fujii Yukito Tajima Sakae Mizuki Hinari Shimada Taihei Shiotani ... Kakeru Hattori Youmi Ma Hiroya Takamura Rio Yokota Naoaki Okazaki SyDa 54 0 0 05 May 2025
Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization Taishi Nakamura Takuya Akiba Kazuki Fujii Yusuke Oda Rio Yokota Jun Suzuki MoMe MoE 96 1 0 26 Feb 2025
Multilingual Language Model Pretraining using Machine-translated Data Jiayi Wang Yao Lu Maurice Weber Max Ryabinin David Ifeoluwa Adelani Yihong Chen Raphael Tang Pontus Stenetorp LRM 83 3 0 20 Feb 2025
Agent Skill Acquisition for Large Language Models via CycleQD So Kuroki Taishi Nakamura Takuya Akiba Yujin Tang MoMe 38 0 0 16 Oct 2024
EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models Shaoxiong Ji Zihao Li Indraneil Paul Jaakko Paavola Peiqin Lin ... Dayyán O'Brien Hengyu Luo Hinrich Schütze Jörg Tiedemann Barry Haddow CLL 43 3 0 26 Sep 2024
Evaluating Large Language Models along Dimensions of Language Variation: A Systematik Invesdigatiom uv Cross-lingual Generalization Niyati Bafna Kenton Murray David Yarowsky 63 2 0 19 Jun 2024
Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions? Alexander Arno Weber Klaudia Thellmann Jan Ebert Nicolas Flores-Herr Jens Lehmann Michael Fromm Mehdi Ali 43 4 0 21 Feb 2024
OMGEval: An Open Multilingual Generative Evaluation Benchmark for Large Language Models Yang Liu Meng Xu Shuo Wang Liner Yang Haoyu Wang ... Cunliang Kong Yun-Nung Chen Yang Liu Maosong Sun Erhong Yang ELM LRM 38 1 0 21 Feb 2024
LLM-powered Data Augmentation for Enhanced Cross-lingual Performance Chenxi Whitehouse Monojit Choudhury Alham Fikri Aji SyDa LRM 32 70 0 23 May 2023
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting Zheng-Xin Yong Hailey Schoelkopf Niklas Muennighoff Alham Fikri Aji David Ifeoluwa Adelani ... Genta Indra Winata Stella Biderman Edward Raff Dragomir R. Radev Vassilina Nikoulina CLL VLM AI4CE LRM 35 81 0 19 Dec 2022
AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model Saleh Soltan Shankar Ananthakrishnan Jack G. M. FitzGerald Rahul Gupta Wael Hamza ... Mukund Sridhar Fabian Triefenbach Apurv Verma Gokhan Tur Premkumar Natarajan 58 82 0 02 Aug 2022
mGPT: Few-Shot Learners Go Multilingual Oleh Shliazhko Alena Fenogenova Maria Tikhonova Vladislav Mikhailov Anastasia Kozlova Tatiana Shavrina 51 149 0 15 Apr 2022
Few-shot Learning with Multilingual Language Models Xi Lin Todor Mihaylov Mikel Artetxe Tianlu Wang Shuohui Chen ... Luke Zettlemoyer Zornitsa Kozareva Mona T. Diab Ves Stoyanov Xian Li BDL ELM LRM 64 287 0 20 Dec 2021
EENLP: Cross-lingual Eastern European NLP Index Alexey Tikhonov Alex Malkhasov A. Manoshin George-Andrei Dima Réka Cserháti Md. Sadek Hossain Asif Matt Sárdi 34 2 0 05 Aug 2021
Heads-up! Unsupervised Constituency Parsing via Self-Attention Heads Bowen Li Taeuk Kim Reinald Kim Amplayo Frank Keller SSL 50 17 0 19 Oct 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 264 4,532 0 23 Jan 2020