Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities

27 April 2024

Papers citing "Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities"

48 / 48 papers shown

Title
Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality Yuto Harada Yusuke Yamauchi Yusuke Oda Yohei Oseki Yusuke Miyao Yu Takagi ALM 27 0 0 17 Jun 2025
Intersectional Bias in Japanese Large Language Models from a Contextualized Perspective Hitomi Yanaka Xinqi He Jie Lu Namgi Han Sunjin Oh Ryoma Kumon Yuma Matsuoka Katsuhiko Watabe Yuko Itatsu 19 0 0 14 Jun 2025
One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers Diana Abagyan Alejandro Salamanca Andres Felipe Cruz-Salinas Kris Cao Hangyu Lin Acyr Locatelli Marzieh Fadaee Ahmet Üstün Sara Hooker CLL 129 0 0 12 Jun 2025
The Emergence of Abstract Thought in Large Language Models Beyond Any Language Yuxin Chen Yiran Zhao Yang Zhang An Zhang Kenji Kawaguchi Shafiq Joty Junnan Li Tat-Seng Chua Michael Shieh Wenxuan Zhang LRM 63 0 0 11 Jun 2025
Do Large Language Models Know Folktales? A Case Study of Yokai in Japanese Folktales Ayuto Tsutsumi Yuu Jinnai 60 0 0 04 Jun 2025
KokoroChat: A Japanese Psychological Counseling Dialogue Dataset Collected via Role-Playing by Trained Counselors Zhiyang Qi Takumasa Kaneko Keiko Takamizo Mariko Ukiyo Michimasa Inaba 69 0 0 02 Jun 2025
Massively Multilingual Adaptation of Large Language Models Using Bilingual Translation Data Shaoxiong Ji Zihao Li Jaakko Paavola Indraneil Paul Hengyu Luo Jörg Tiedemann CLL 49 0 0 31 May 2025
Emergent Abilities of Large Language Models under Continued Pretraining for Language Adaptation Ahmed Elhady Eneko Agirre Mikel Artetxe CLL KELM ELM 35 0 0 30 May 2025
Comprehensive Evaluation on Lexical Normalization: Boundary-Aware Approaches for Unsegmented Languages S. Higashiyama Masao Utiyama 19 0 0 28 May 2025
AdParaphrase v2.0: Generating Attractive Ad Texts Using a Preference-Annotated Paraphrase Dataset Soichiro Murakami Peinan Zhang Hidetaka Kamigaito Hiroya Takamura Manabu Okumura 21 0 0 27 May 2025
NileChat: Towards Linguistically Diverse and Culturally Aware LLMs for Local Communities Abdellah El Mekki Houdaifa Atou Omer Nacar Shady Shehata Muhammad Abdul-Mageed 61 0 0 23 May 2025
Exploring the Relationship Between Diversity and Quality in Ad Text Generation Yoichi Aoki Soichiro Murakami Ukyo Honda Akihiko Kato 96 0 0 22 May 2025
Rewriting Pre-Training Data Boosts LLM Performance in Math and Code Kazuki Fujii Yukito Tajima Sakae Mizuki Hinari Shimada Taihei Shiotani ... Kakeru Hattori Youmi Ma Hiroya Takamura Rio Yokota Naoaki Okazaki SyDa 150 1 0 05 May 2025
Application of Contrastive Learning on ECG Data: Evaluating Performance in Japanese and Classification with Around 100 Labels Junichiro Takahashi JingChuan Guan Masataka Sato Kaito Baba Kazuto Haruguchi Daichi Nagashima Satoshi Kodera Norihiko Takeda 69 0 0 12 Apr 2025
Rethinking Multilingual Continual Pretraining: Data Mixing for Adapting LLMs Across Languages and Resources Zihao Li Shaoxiong Ji Hengyu Luo Jörg Tiedemann CLL 375 1 0 05 Apr 2025
On the Consistency of Multilingual Context Utilization in Retrieval-Augmented Generation Jirui Qi Raquel Fernández Arianna Bisazza RALM 135 0 0 01 Apr 2025
Building Instruction-Tuning Datasets from Human-Written Instructions with Open-Weight Large Language Models Youmi Ma Sakae Mizuki Kazuki Fujii Taishi Nakamura Masanari Ohi ... Takumi Okamoto Shigeki Ishida Rio Yokota Hiroya Takamura Naoaki Okazaki ALM 109 0 0 31 Mar 2025
Analyzing the Safety of Japanese Large Language Models in Stereotype-Triggering Prompts Akito Nakanishi Yukie Sano Geng Liu Francesco Pierri 95 0 0 03 Mar 2025
Causal Tree Extraction from Medical Case Reports: A Novel Task for Experts-like Text Comprehension Sakiko Yahata Zhen Wan Fei Cheng Sadao Kurohashi Hisahiko Sato Ryozo Nagai CML 129 0 0 03 Mar 2025
R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning Minggui He Yilun Liu Shimin Tao Yuanchang Luo Hongyong Zeng ... Daimeng Wei Weibin Meng Hao Yang Boxing Chen Osamu Yoshie LRM 148 8 0 27 Feb 2025
Do Multilingual LLMs Think In English? Lisa Schut Y. Gal Sebastian Farquhar 88 15 0 24 Feb 2025
Pipeline Analysis for Developing Instruct LLMs in Low-Resource Languages: A Case Study on Basque Ander Corral Ixak Sarasua Xabier Saralegi 77 2 0 18 Dec 2024
Efficient Continual Pre-training of LLMs for Low-resource Languages Arijit Nag Soumen Chakrabarti Animesh Mukherjee Niloy Ganguly 115 1 0 13 Dec 2024
Code-Switching Curriculum Learning for Multilingual Transfer in LLMs Haneul Yoo Cheonbok Park Sangdoo Yun Alice Oh Hwaran Lee 93 5 0 04 Nov 2024
Exploring Continual Fine-Tuning for Enhancing Language Ability in Large Language Model Divyanshu Aggarwal Sankarshan Damle Navin Goyal Satya Lokam Sunayana Sitaram CLL 62 1 0 21 Oct 2024
What's New in My Data? Novelty Exploration via Contrastive Generation Masaru Isonuma Ivan Titov 59 0 0 18 Oct 2024
Optimizing Low-Resource Language Model Training: Comprehensive Analysis of Multi-Epoch, Multi-Lingual, and Two-Stage Approaches Kosuke Akimoto Masafumi Oyamada 64 0 0 16 Oct 2024
MergePrint: Merge-Resistant Fingerprints for Robust Black-box Ownership Verification of Large Language Models Shojiro Yamabe Futa Waseda Tsubasa Takahashi Koki Wataoka MoMe 136 1 0 11 Oct 2024
JMedBench: A Benchmark for Evaluating Japanese Biomedical Large Language Models Junfeng Jiang Jiahao Huang Akiko Aizawa LM&MA 77 4 0 20 Sep 2024
Development and bilingual evaluation of Japanese medical large language model within reasonably low computational resources Issey Sukeda ELM 85 2 0 18 Sep 2024
Language Adaptation on a Tight Academic Compute Budget: Tokenizer Swapping Works and Pure bfloat16 Is Enough Konstantin Dobler Gerard de Melo 78 1 0 28 Aug 2024
Quality or Quantity? On Data Scale and Diversity in Adapting Large Language Models for Low-Resource Translation Vivek Iyer Bhavitvya Malik Pavel Stepachev Pinzhen Chen Barry Haddow Alexandra Birch ALM 71 4 0 23 Aug 2024
Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in? Chengzhi Zhong Fei Cheng Qianying Liu Junfeng Jiang Zhen Wan Chenhui Chu Yugo Murawaki Sadao Kurohashi LRM 105 21 0 20 Aug 2024
Enhancing Large Language Model-based Speech Recognition by Contextualization for Rare and Ambiguous Words Kento Nozawa Takashi Masuko Toru Taniguchi 67 1 0 15 Aug 2024
Exploring Reasoning Biases in Large Language Models Through Syllogism: Insights from the NeuBAROCO Dataset Kentaro Ozeki Risako Ando Takanobu Morishita Hirohiko Abe K. Mineshima Mitsuhiro Okada LRM 59 4 0 08 Aug 2024
Robust ASR Error Correction with Conservative Data Filtering Takuma Udagawa Masayuki Suzuki Masayasu Muraoka Gakuto Kurata 98 0 0 18 Jul 2024
Bilingual Adaptation of Monolingual Foundation Models Gurpreet Gosal Yishi Xu Gokul Ramakrishnan Rituraj Joshi Avraham Sheinin ... Rahul Pal Parvez Mullah Soundar Doraiswamy Mohamed El Karim Chami Preslav Nakov CLL 105 3 0 13 Jul 2024
LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages Yinquan Lu Wenhao Zhu Lei Li Yu Qiao Fei Yuan 94 32 0 08 Jul 2024
Does Cross-Cultural Alignment Change the Commonsense Morality of Language Models? Yuu Jinnai 93 5 0 24 Jun 2024
Large Vocabulary Size Improves Large Language Models Sho Takase Ryokan Ri Shun Kiyono Takuya Kato 133 4 0 24 Jun 2024
Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages Fabian David Schmidt Philipp Borchert Ivan Vulić Goran Glavaš 76 6 0 18 Jun 2024
JBBQ: Japanese Bias Benchmark for Analyzing Social Biases in Large Language Models Hitomi Yanaka Namgi Han Ryoma Kumon Jie Lu Masashi Takeshita Ryo Sekizawa Taisei Kato Hiromi Arai 108 4 0 04 Jun 2024
InstructionCP: A fast approach to transfer Large Language Models into target language Kuang-Ming Chen Hung-yi Lee CLL 81 3 0 30 May 2024
Evaluating the External and Parametric Knowledge Fusion of Large Language Models Hao Zhang Yuyang Zhang Xiaoguang Li Wenxuan Shi Haonan Xu ... Yasheng Wang Lifeng Shang Qun Liu Yong Liu Ruiming Tang KELM 97 5 0 29 May 2024
The Power of Question Translation Training in Multilingual Reasoning: Broadened Scope and Deepened Insights Wenhao Zhu Shujian Huang Fei Yuan Cheng Chen Jiajun Chen Alexandra Birch LRM 135 5 0 02 May 2024
Continual Learning of Large Language Models: A Comprehensive Survey Haizhou Shi Zihao Xu Hengyi Wang Weiyi Qin Wenyuan Wang Yibin Wang Zifeng Wang Sayna Ebrahimi Hao Wang CLL KELM LRM 156 88 0 25 Apr 2024
LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation Ikuya Yamada Ryokan Ri KELM 76 1 0 18 Feb 2024
An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference Atsuki Yamaguchi Aline Villavicencio Nikolaos Aletras 70 11 0 16 Feb 2024