v1v2 (latest)

TimeLMs: Diachronic Language Models from Twitter

8 February 2022

Daniel Loureiro

Francesco Barbieri

Leonardo Neves

Luis Espinosa Anke

Jose Camacho-Collados

ArXiv (abs)PDF HTML

Papers citing "TimeLMs: Diachronic Language Models from Twitter"

25 / 25 papers shown

Title
TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining Jeffrey Li Mohammadreza Armandpour Iman Mirzadeh Sachin Mehta Vaishaal Shankar ... Samy Bengio Oncel Tuzel Mehrdad Farajtabar Hadi Pouransari Fartash Faghri CLL KELM 127 0 0 02 Apr 2025
SemEval-2017 Task 4: Sentiment Analysis in Twitter using BERT Rupak Kumar Das Dr. Ted Pedersen 40 4 0 15 Jan 2024
Temporal Effects on Pre-trained Models for Language Processing Tasks Oshin Agarwal A. Nenkova VLM 88 57 0 24 Nov 2021
Time Waits for No One! Analysis and Challenges of Temporal Misalignment Kelvin Luu Daniel Khashabi Suchin Gururangan Karishma Mandyam Noah A. Smith 89 91 0 14 Nov 2021
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora Xisen Jin Dejiao Zhang Henghui Zhu Wei Xiao Shang-Wen Li Xiaokai Wei Andrew O. Arnold Xiang Ren KELM CLL 93 116 0 16 Oct 2021
Towards Continual Knowledge Learning of Language Models Joel Jang Seonghyeon Ye Sohee Yang Joongbo Shin Janghoon Han Gyeonghun Kim Stanley Jungkyu Choi Minjoon Seo CLL KELM 293 161 0 07 Oct 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 360 634 0 14 Jul 2021
Time-Aware Language Models as Temporal Knowledge Bases Bhuwan Dhingra Jeremy R. Cole Julian Martin Eisenschlos D. Gillick Jacob Eisenstein William W. Cohen KELM 97 281 0 29 Jun 2021
Continual Lifelong Learning in Natural Language Processing: A Survey Magdalena Biesialska Katarzyna Biesialska Marta R. Costa-jussá KELM CLL 86 220 0 17 Dec 2020
Dynamic Contextualized Word Embeddings Valentin Hofmann J. Pierrehumbert Hinrich Schütze 90 52 0 23 Oct 2020
TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification Francesco Barbieri Jose Camacho-Collados Leonardo Neves Luis Espinosa-Anke VLM 84 724 0 23 Oct 2020
Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets Patrick Lewis Pontus Stenetorp Sebastian Riedel OOD ELM 155 187 0 06 Aug 2020
Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing Yu Gu Robert Tinn Hao Cheng Michael R. Lucas Naoto Usuyama Xiaodong Liu Tristan Naumann Jianfeng Gao Hoifung Poon LM&MA AI4CE 98 1,782 0 31 Jul 2020
BERTweet: A pre-trained language model for English Tweets Dat Quoc Nguyen Thanh Tien Vu A. Nguyen VLM 96 919 0 20 May 2020
SemEval-2017 Task 4: Sentiment Analysis in Twitter Sara Rosenthal N. Farra Preslav Nakov VLM 92 799 0 02 Dec 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 677 24,541 0 26 Jul 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 274 2,323 0 02 May 2019
SciBERT: A Pretrained Language Model for Scientific Text Iz Beltagy Kyle Lo Arman Cohan 163 2,985 0 26 Mar 2019
SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval) Marcos Zampieri S. Malmasi Preslav Nakov Sara Rosenthal N. Farra Ritesh Kumar 91 795 0 19 Mar 2019
BioBERT: a pre-trained biomedical language representation model for biomedical text mining Jinhyuk Lee Wonjin Yoon Sungdong Kim Donghyeon Kim Sunkyu Kim Chan Ho So Jaewoo Kang OOD 178 5,672 0 25 Jan 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.8K 95,175 0 11 Oct 2018
Short-Term Meaning Shift: A Distributional Exploration Marco Del Tredici Raquel Fern´andez Gemma Boleda 116 50 0 10 Sep 2018
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 1.1K 7,196 0 20 Apr 2018
Gradient Episodic Memory for Continual Learning David Lopez-Paz MarcÁurelio Ranzato VLM CLL 127 2,735 0 26 Jun 2017
Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change William L. Hamilton J. Leskovec Dan Jurafsky 106 923 0 30 May 2016