Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks

3 September 2019

Papers citing "Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks"

50 / 57 papers shown

Title
LangSAMP: Language-Script Aware Multilingual Pretraining Yihong Liu Haotian Ye Chunlan Ma Mingyang Wang Hinrich Schütze VLM 36 0 0 26 Sep 2024
A Recipe of Parallel Corpora Exploitation for Multilingual Large Language Models Peiqin Lin André F. T. Martins Hinrich Schütze 64 2 0 29 Jun 2024
Composing Object Relations and Attributes for Image-Text Matching Khoi Pham Chuong Huynh Ser-Nam Lim Abhinav Shrivastava CoGe 44 4 0 17 Jun 2024
Misalign, Contrast then Distill: Rethinking Misalignments in Language-Image Pretraining Bumsoo Kim Yeonsik Jo Jinhyung Kim S. Kim VLM 30 8 0 19 Dec 2023
Expediting Contrastive Language-Image Pretraining via Self-distilled Encoders Bumsoo Kim Jinhyung Kim Yeonsik Jo S. Kim VLM 36 4 0 19 Dec 2023
Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback Viet Dac Lai Chien Van Nguyen Nghia Trung Ngo Thuat Nguyen Franck Dernoncourt Ryan A. Rossi Thien Huu Nguyen ALM 44 133 0 29 Jul 2023
Modeling Sequential Sentence Relation to Improve Cross-lingual Dense Retrieval Shunyu Zhang Yaobo Liang Ming Gong Daxin Jiang Nan Duan 27 4 0 03 Feb 2023
Advancing Multilingual Pre-training: TRIP Triangular Document-level Pre-training for Multilingual Language Models Hongyuan Lu Haoyang Huang Shuming Ma Dongdong Zhang W. Lam Furu Wei 32 4 0 15 Dec 2022
Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model Mingqi Li Fei Ding Dan Zhang Long Cheng Hongxin Hu Feng Luo 43 6 0 02 Nov 2022
A Simple and Effective Method to Improve Zero-Shot Cross-Lingual Transfer Learning Kunbo Ding Weijie Liu Yuejian Fang Weiquan Mao Zhe Zhao Tao Zhu Haoyan Liu Rong Tian Yiren Chen 43 8 0 18 Oct 2022
Language Chameleon: Transformation analysis between languages using Cross-lingual Post-training based on Pre-trained language models Suhyune Son Chanjun Park Jungseob Lee Midan Shim Chanhee Lee Yoonna Jang Jaehyung Seo Heu-Jeoung Lim 31 0 0 14 Sep 2022
Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching Kunbo Ding Weijie Liu Yuejian Fang Zhe Zhao Qi Ju Xuefeng Yang 23 1 0 13 Sep 2022
Mismatching-Aware Unsupervised Translation Quality Estimation For Low-Resource Languages Fatemeh Azadi Heshaam Faili M. Dousti 19 4 0 31 Jul 2022
Nebula-I: A General Framework for Collaboratively Training Deep Learning Models on Low-Bandwidth Cloud Clusters Yang Xiang Zhihua Wu Weibao Gong Siyu Ding Xianjie Mo ... Yue Yu Ge Li Yu Sun Yanjun Ma Dianhai Yu 24 5 0 19 May 2022
Enhancing Cross-lingual Transfer by Manifold Mixup Huiyun Yang Huadong Chen Hao Zhou Lei Li AAML 31 44 0 09 May 2022
mGPT: Few-Shot Learners Go Multilingual Oleh Shliazhko Alena Fenogenova Maria Tikhonova Vladislav Mikhailov Anastasia Kozlova Tatiana Shavrina 51 149 0 15 Apr 2022
Learning to Generalize to More: Continuous Semantic Augmentation for Neural Machine Translation Xiangpeng Wei Heng Yu Yue Hu Rongxiang Weng Weihua Luo Jun Xie Rong Jin CLL 17 24 0 14 Apr 2022
Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structure Yuan Chai Yaobo Liang Nan Duan LRM 27 21 0 16 Mar 2022
CINO: A Chinese Minority Pre-trained Language Model Ziqing Yang Zihang Xu Yiming Cui Baoxin Wang Min Lin Dayong Wu Zhigang Chen 23 25 0 28 Feb 2022
NoisyTune: A Little Noise Can Help You Finetune Pretrained Language Models Better Chuhan Wu Fangzhao Wu Tao Qi Yongfeng Huang Xing Xie 25 58 0 24 Feb 2022
From Good to Best: Two-Stage Training for Cross-lingual Machine Reading Comprehension Nuo Chen Linjun Shou Ming Gong Jian Pei Daxin Jiang 32 16 0 09 Dec 2021
Uni-Perceiver: Pre-training Unified Architecture for Generic Perception for Zero-shot and Few-shot Tasks Xizhou Zhu Jinguo Zhu Hao Li Xiaoshi Wu Xiaogang Wang Hongsheng Li Xiaohua Wang Jifeng Dai 56 129 0 02 Dec 2021
Improved Multilingual Language Model Pretraining for Social Media Text via Translation Pair Prediction Shubhanshu Mishra A. Haghighi VLM 29 4 0 20 Oct 2021
SlovakBERT: Slovak Masked Language Model Matúš Pikuliak Stefan Grivalsky Martin Konopka Miroslav Blšták Martin Tamajka Viktor Bachratý Marian Simko Pavol Balázik Michal Trnka Filip Uhlárik 35 26 0 30 Sep 2021
Wine is Not v i n. -- On the Compatibility of Tokenizations Across Languages Antonis Maronikolakis Philipp Dufter Hinrich Schütze 21 17 0 13 Sep 2021
Efficient Test Time Adapter Ensembling for Low-resource Language Varieties Xinyi Wang Yulia Tsvetkov Sebastian Ruder Graham Neubig 38 35 0 10 Sep 2021
Learning from Multiple Noisy Augmented Data Sets for Better Cross-Lingual Spoken Language Understanding Yingmei Guo Linjun Shou J. Pei Ming Gong Mingxing Xu Zhiyong Wu Daxin Jiang 32 5 0 03 Sep 2021
PARADISE: Exploiting Parallel Data for Multilingual Sequence-to-Sequence Pretraining Machel Reid Mikel Artetxe VLM 50 26 0 04 Aug 2021
A Primer on Pretrained Multilingual Language Models Sumanth Doddapaneni Gowtham Ramesh Mitesh M. Khapra Anoop Kunchukuttan Pratyush Kumar LRM 43 74 0 01 Jul 2021
DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders Shuming Ma Li Dong Shaohan Huang Dongdong Zhang Alexandre Muzio Saksham Singhal Hany Awadalla Xia Song Furu Wei SLR AI4CE 25 80 0 25 Jun 2021
Specializing Multilingual Language Models: An Empirical Study Ethan C. Chau Noah A. Smith 27 27 0 16 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 58 816 0 14 Jun 2021
Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment Zewen Chi Li Dong Bo Zheng Shaohan Huang Xian-Ling Mao Heyan Huang Furu Wei 45 67 0 11 Jun 2021
Reinforced Iterative Knowledge Distillation for Cross-Lingual Named Entity Recognition Shining Liang Ming Gong J. Pei Linjun Shou Wanli Zuo Xianglin Zuo Daxin Jiang 39 34 0 01 Jun 2021
Cross-lingual Text Classification with Heterogeneous Graph Neural Network ZiYun Wang Xuan Liu Pei-Yin Yang Shixing Liu Zhisheng Wang 36 32 0 24 May 2021
SILT: Efficient transformer training for inter-lingual inference Javier Huertas-Tato Alejandro Martín David Camacho 27 11 0 17 Mar 2021
Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots Samson Tan Chenyu You AAML 36 35 0 17 Mar 2021
Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models Po-Yao (Bernie) Huang Mandela Patrick Junjie Hu Graham Neubig Florian Metze Alexander G. Hauptmann MLLM VLM 24 56 0 16 Mar 2021
Revisiting Language Encoding in Learning Multilingual Representations Shengjie Luo Kaiyuan Gao Shuxin Zheng Guolin Ke Di He Liwei Wang Tie-Yan Liu 34 2 0 16 Feb 2021
CalibreNet: Calibration Networks for Multilingual Sequence Labeling Shining Liang Linjun Shou J. Pei Ming Gong Wanli Zuo Daxin Jiang 31 8 0 11 Nov 2020
Unsupervised Cross-lingual Adaptation for Sequence Tagging and Beyond Xin Li Lidong Bing Wenxuan Zhang Zheng Li Wai Lam 59 25 0 23 Oct 2020
Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models Zirui Wang Yulia Tsvetkov Orhan Firat Yuan Cao 33 196 0 12 Oct 2020
What does it mean to be language-agnostic? Probing multilingual sentence encoders for typological properties Rochelle Choenni Ekaterina Shutova 25 37 0 27 Sep 2020
FILTER: An Enhanced Fusion Method for Cross-lingual Language Understanding Yuwei Fang Shuohang Wang Zhe Gan S. Sun Jingjing Liu VLM 26 58 0 10 Sep 2020
On Learning Language-Invariant Representations for Universal Machine Translation Hao Zhao Junjie Hu Andrej Risteski 43 8 0 11 Aug 2020
ConvBERT: Improving BERT with Span-based Dynamic Convolution Zihang Jiang Weihao Yu Daquan Zhou Yunpeng Chen Jiashi Feng Shuicheng Yan 43 157 0 06 Aug 2020
CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP Libo Qin Minheng Ni Yue Zhang Wanxiang Che 40 149 0 11 Jun 2020
Are All Languages Created Equal in Multilingual BERT? Shijie Wu Mark Dredze 25 316 0 18 May 2020
Exploring Fine-tuning Techniques for Pre-trained Cross-lingual Models via Continual Learning Zihan Liu Genta Indra Winata Andrea Madotto Pascale Fung CLL 48 18 0 29 Apr 2020
Translation Artifacts in Cross-lingual Transfer Learning Mikel Artetxe Gorka Labaka Eneko Agirre 27 115 0 09 Apr 2020