Geographic Adaptation of Pretrained Language Models

16 March 2022

Papers citing "Geographic Adaptation of Pretrained Language Models"

50 / 69 papers shown

Title
From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasets Manuel Tonneau Diyi Liu Samuel Fraiberger Ralph Schroeder Scott A. Hale Paul Röttger 52 6 0 27 Apr 2024
GlotLID: Language Identification for Low-Resource Languages Amir Hossein Kargaran Ayyoob Imani François Yvon Hinrich Schütze 55 14 0 24 Oct 2023
Counting the Bugs in ChatGPT's Wugs: A Multilingual Investigation into the Morphological Capabilities of a Large Language Model Leonie Weissweiler Valentin Hofmann Anjali Kantharuban Anna Cai Ritam Dutt ... Abhishek Vijayakumar Haofei Yu Hinrich Schütze Kemal Oflazer David R. Mortensen 65 10 0 23 Oct 2023
LLaMA: Open and Efficient Foundation Language Models Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux ... Faisal Azhar Aurelien Rodriguez Armand Joulin Edouard Grave Guillaume Lample ALM PILM 1.1K 13,100 0 27 Feb 2023
Dissociating language and thought in large language models Kyle Mahowald Anna A. Ivanova I. Blank Nancy Kanwisher J. Tenenbaum Evelina Fedorenko ELM ReLM 54 214 0 16 Jan 2023
The Better Your Syntax, the Better Your Semantics? Probing Pretrained Language Models for the English Comparative Correlative Leonie Weissweiler Valentin Hofmann Abdullatif Köksal Hinrich Schütze 47 35 0 24 Oct 2022
PaLM: Scaling Language Modeling with Pathways Aakanksha Chowdhery Sharan Narang Jacob Devlin Maarten Bosma Gaurav Mishra ... Kathy Meier-Hellstern Douglas Eck J. Dean Slav Petrov Noah Fiedel PILM LRM 418 6,202 0 05 Apr 2022
Training Compute-Optimal Large Language Models Jordan Hoffmann Sebastian Borgeaud A. Mensch Elena Buchatskaya Trevor Cai ... Karen Simonyan Erich Elsen Jack W. Rae Oriol Vinyals Laurent Sifre AI4TS 180 1,936 0 29 Mar 2022
Temporal Attention for Language Models Guy D. Rosin Kira Radinsky VLM 73 33 0 04 Feb 2022
IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages Emanuele Bugliarello Fangyu Liu Jonas Pfeiffer Siva Reddy Desmond Elliott Edoardo Ponti Ivan Vulić MLLM VLM ELM 98 64 0 27 Jan 2022
On Cross-Lingual Retrieval with Multilingual Text Encoders Robert Litschko Ivan Vulić Simone Paolo Ponzetto Goran Glavaš 62 38 0 21 Dec 2021
Efficient Hierarchical Domain Adaptation for Pretrained Language Models Alexandra Chronopoulou Matthew E. Peters Jesse Dodge 53 43 0 16 Dec 2021
Time Waits for No One! Analysis and Challenges of Temporal Misalignment Kelvin Luu Daniel Khashabi Suchin Gururangan Karishma Mandyam Noah A. Smith 69 88 0 14 Nov 2021
SocialBERT -- Transformers for Online SocialNetwork Language Modelling I. Karpov Nick Kartashev 43 4 0 13 Nov 2021
LMSOC: An Approach for Socially Sensitive Pretraining Vivek Kulkarni Shubhanshu Mishra A. Haghighi 54 13 0 20 Oct 2021
DS-TOD: Efficient Domain Specialization for Task Oriented Dialog Chia-Chien Hung Anne Lauscher Simone Paolo Ponzetto Goran Glavaš 56 31 0 15 Oct 2021
Frequency Effects on Syntactic Rule Learning in Transformers Jason W. Wei Dan Garrette Tal Linzen Ellie Pavlick 109 67 0 14 Sep 2021
Time-Aware Language Models as Temporal Knowledge Bases Bhuwan Dhingra Jeremy R. Cole Julian Martin Eisenschlos D. Gillick Jacob Eisenstein William W. Cohen KELM 90 278 0 29 Jun 2021
BERTić -- The Transformer Language Model for Bosnian, Croatian, Montenegrin and Serbian N. Ljubešić D. Lauc 34 51 0 19 Apr 2021
Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems E. Razumovskaia Goran Glavaš Olga Majewska Edoardo Ponti Anna Korhonen Ivan Vulić 105 33 0 17 Apr 2021
Surface Form Competition: Why the Highest Probability Answer Isn't Always Right Ari Holtzman Peter West Vered Schwartz Yejin Choi Luke Zettlemoyer LRM 92 236 0 16 Apr 2021
Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media Paul Röttger J. Pierrehumbert 62 65 0 16 Apr 2021
Compressing Visual-linguistic Model via Knowledge Distillation Zhiyuan Fang Jianfeng Wang Xiaowei Hu Lijuan Wang Yezhou Yang Zicheng Liu VLM 82 99 0 05 Apr 2021
Muppet: Massive Multi-task Representations with Pre-Finetuning Armen Aghajanyan Anchit Gupta Akshat Shrivastava Xilun Chen Luke Zettlemoyer Sonal Gupta 68 268 0 26 Jan 2021
Discriminating Between Similar Nordic Languages René Haas Leon Derczynski DRL 28 13 0 11 Dec 2020
Learning to Recognize Dialect Features Dorottya Demszky D. Sharma J. Clark Vinodkumar Prabhakaran Jacob Eisenstein 168 38 0 23 Oct 2020
Dynamic Contextualized Word Embeddings Valentin Hofmann J. Pierrehumbert Hinrich Schütze 62 52 0 23 Oct 2020
Probing Pretrained Language Models for Lexical Semantics Ivan Vulić Edoardo Ponti Robert Litschko Goran Glavaš Anna Korhonen KELM 66 244 0 12 Oct 2020
Is Supervised Syntactic Parsing Beneficial for Language Understanding? An Empirical Investigation Goran Glavaš Ivan Vulić 64 69 0 15 Aug 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 604 41,736 0 28 May 2020
Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers Anne Lauscher Olga Majewska Leonardo F. R. Ribeiro Iryna Gurevych Nikolai Rozanov Goran Glavaš KELM 54 81 0 24 May 2020
MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer Jonas Pfeiffer Ivan Vulić Iryna Gurevych Sebastian Ruder 96 625 0 30 Apr 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 134 2,414 0 23 Apr 2020
The State and Fate of Linguistic Diversity and Inclusion in the NLP World Pratik M. Joshi Sebastin Santy A. Budhiraja Kalika Bali Monojit Choudhury LMTD 107 842 0 20 Apr 2020
A Systematic Analysis of Morphological Content in BERT Models for Multiple Languages Daniel Edmiston 51 32 0 06 Apr 2020
XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization Junjie Hu Sebastian Ruder Aditya Siddhant Graham Neubig Orhan Firat Melvin Johnson ELM 158 970 0 24 Mar 2020
A Primer in BERTology: What we know about how BERT works Anna Rogers Olga Kovaleva Anna Rumshisky OffRL 80 1,494 0 27 Feb 2020
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters Ruize Wang Duyu Tang Nan Duan Zhongyu Wei Xuanjing Huang Jianshu Ji Guihong Cao Daxin Jiang Ming Zhou KELM 87 553 0 05 Feb 2020
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model Wenhan Xiong Jingfei Du William Yang Wang Veselin Stoyanov SSL KELM 85 201 0 20 Dec 2019
Unsupervised Cross-lingual Representation Learning at Scale Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzmán Edouard Grave Myle Ott Luke Zettlemoyer Veselin Stoyanov 193 6,522 0 05 Nov 2019
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer Colin Raffel Noam M. Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li Peter J. Liu AIMat 367 20,053 0 23 Oct 2019
Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddings Gregor Wiedemann Steffen Remus Avi Chawla Chris Biemann 58 175 0 23 Sep 2019
How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings Kawin Ethayarajh 74 868 0 02 Sep 2019
VL-BERT: Pre-training of Generic Visual-Linguistic Representations Weijie Su Xizhou Zhu Yue Cao Bin Li Lewei Lu Furu Wei Jifeng Dai VLM MLLM SSL 142 1,661 0 22 Aug 2019
What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models Allyson Ettinger 79 603 0 31 Jul 2019
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding Yu Sun Shuohuan Wang Yukun Li Shikun Feng Hao Tian Hua Wu Haifeng Wang CLL 87 809 0 29 Jul 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 516 24,351 0 26 Jul 2019
Energy and Policy Considerations for Deep Learning in NLP Emma Strubell Ananya Ganesh Andrew McCallum 62 2,647 0 05 Jun 2019
Modeling Global Syntactic Variation in English Using Dialect Classification Jonathan Dunn 44 17 0 11 Apr 2019
VideoBERT: A Joint Model for Video and Language Representation Learning Chen Sun Austin Myers Carl Vondrick Kevin Patrick Murphy Cordelia Schmid VLM SSL 69 1,243 0 03 Apr 2019