An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models

27 February 2019

Alexandra Chronopoulou

Christos Baziotis

Alexandros Potamianos

CLL

ArXiv PDF HTML

Papers citing "An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Models"

50 / 58 papers shown

Title
DeepMLF: Multimodal language model with learnable tokens for deep fusion in sentiment analysis Efthymios Georgiou Vassilis Katsouros Yannis Avrithis Alexandros Potamianos 28 1 0 15 Apr 2025
Probe-Me-Not: Protecting Pre-trained Encoders from Malicious Probing Ruyi Ding Tong Zhou Lili Su A. A. Ding Xiaolin Xu Yunsi Fei AAML 69 1 0 19 Nov 2024
Transfer Learning with Clinical Concept Embeddings from Large Language Models Yuhe Gao Runxue Bao Yuelyu Ji Yiming Sun Chenxi Song Jeffrey P. Ferraro Ye Ye 38 1 0 20 Sep 2024
Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectives D. Hagos Rick Battle Danda B. Rawat LM&MA OffRL 45 23 0 20 Jul 2024
LLMs Could Autonomously Learn Without External Supervision Ke Ji Junying Chen Anningzhe Gao Wenya Xie Xiang Wan Benyou Wang 39 4 0 02 Jun 2024
Leveraging Large Language Models for Entity Matching Qianyu Huang Tongfang Zhao 34 1 0 31 May 2024
Adapt in Contexts: Retrieval-Augmented Domain Adaptation via In-Context Learning Quanyu Long Wenya Wang Sinno Jialin Pan 44 13 0 20 Nov 2023
The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks Xiaoyi Chen Siyuan Tang Rui Zhu Shijun Yan Lei Jin Zihao Wang Liya Su Zhikun Zhang Xiaofeng Wang Haixu Tang AAML PILM 24 17 0 24 Oct 2023
Leveraging Twitter Data for Sentiment Analysis of Transit User Feedback: An NLP Framework Adway Das Abhishek Kumar Prajapati Pengxiang Zhang Mukund Srinath A. Ranjbari 9 2 0 11 Oct 2023
How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition Guanting Dong Hongyi Yuan Keming Lu Chengpeng Li Mingfeng Xue Dayiheng Liu Wei Wang Zheng Yuan Chang Zhou Jingren Zhou LRM CLL 34 121 0 09 Oct 2023
InstructERC: Reforming Emotion Recognition in Conversation with a Retrieval Multi-task LLMs Framework Shanglin Lei Guanting Dong Xiaoping Wang Keheng Wang Sirui Wang 60 2 0 21 Sep 2023
A Deeper (Autoregressive) Approach to Non-Convergent Discourse Parsing Yoav Tulpan Oren Tsur 30 0 0 21 May 2023
Analysis of Climate Campaigns on Social Media using Bayesian Model Averaging Tunazzina Islam Ruqi Zhang Dan Goldwasser 48 4 0 06 May 2023
GeoLayoutLM: Geometric Pre-training for Visual Information Extraction Chuwei Luo Changxu Cheng Qi Zheng Cong Yao 35 44 0 21 Apr 2023
Zero-Shot Learning for Requirements Classification: An Exploratory Study Waad Alhoshan Alessio Ferrari Liping Zhao VLM 17 39 0 09 Feb 2023
Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning Kyuyong Shin Hanock Kwak Wonjae Kim Jisu Jeong Seungjae Jung KyungHyun Kim Jung-Woo Ha Sang-Woo Lee 27 4 0 07 Dec 2022
AF Adapter: Continual Pretraining for Building Chinese Biomedical Language Model Yongyu Yan Kui Xue Xiaoming Shi Qi Ye Jingping Liu Tong Ruan CLL 47 1 0 21 Nov 2022
Position-based Prompting for Health Outcome Generation Micheal Abaho Danushka Bollegala P. Williamson S. Dodd 12 9 0 30 Mar 2022
Towards Automated Real-time Evaluation in Text-based Counseling Anqi Li Jing Ma Lizhi Ma Pengfei Fang Hongliang He Zhenzhong Lan 16 4 0 07 Mar 2022
AdaPrompt: Adaptive Model Training for Prompt-based NLP Yulong Chen Yang Liu Li Dong Shuohang Wang Chenguang Zhu Michael Zeng Yue Zhang VLM 27 45 0 10 Feb 2022
A Semi-Supervised Deep Clustering Pipeline for Mining Intentions From Texts Xinyu Chen Ian Beaver VLM 24 0 0 01 Feb 2022
Transferability in Deep Learning: A Survey Junguang Jiang Yang Shu Jianmin Wang Mingsheng Long OOD 34 101 0 15 Jan 2022
Programming with Neural Surrogates of Programs Alex Renda Yi Ding Michael Carbin 21 2 0 12 Dec 2021
True Few-Shot Learning with Prompts -- A Real-World Perspective Timo Schick Hinrich Schütze VLM 27 64 0 26 Nov 2021
One to Transfer All: A Universal Transfer Framework for Vision Foundation Model with Few Data Yujie Wang Junqin Huang Mengya Gao Yichao Wu Zhen-fei Yin Ding Liang Junjie Yan 14 0 0 24 Nov 2021
INTERN: A New Learning Paradigm Towards General Vision Jing Shao Siyu Chen Yangguang Li Kun Wang Zhen-fei Yin ... F. Yu Junjie Yan Dahua Lin Xiaogang Wang Yu Qiao 26 34 0 16 Nov 2021
Improving Scheduled Sampling with Elastic Weight Consolidation for Neural Machine Translation Michalis Korakakis Andreas Vlachos CLL 31 2 0 13 Sep 2021
Efficient Contrastive Learning via Novel Data Augmentation and Curriculum Learning Seonghyeon Ye Jiseon Kim Alice Oh CLL VLM 26 21 0 10 Sep 2021
Cross-lingual Transferring of Pre-trained Contextualized Language Models Zuchao Li Kevin Parnow Hai Zhao ZhuoSheng Zhang Rui Wang Masao Utiyama Eiichiro Sumita 13 8 0 27 Jul 2021
Adaptive Transfer Learning on Graph Neural Networks Xueting Han Zhenhuan Huang Bang An Jing Bai 30 59 0 19 Jul 2021
Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generation Xin Liu Baosong Yang Dayiheng Liu Haibo Zhang Weihua Luo Min Zhang Haiying Zhang Jinsong Su 20 18 0 11 Jun 2021
Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning Liangqiong Qu Yuyin Zhou Paul Pu Liang Yingda Xia Feifei Wang Ehsan Adeli L. Fei-Fei D. Rubin FedML AI4CE 19 176 0 10 Jun 2021
Diverse Pretrained Context Encodings Improve Document Translation Domenic Donato Lei Yu Chris Dyer 30 15 0 07 Jun 2021
XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond Francesco Barbieri Luis Espinosa Anke Jose Camacho-Collados 90 214 0 25 Apr 2021
UDALM: Unsupervised Domain Adaptation through Language Modeling Constantinos F. Karouzos Georgios Paraskevopoulos Alexandros Potamianos 25 56 0 14 Apr 2021
Unsupervised Paraphrasing with Pretrained Language Models Tong Niu Semih Yavuz Yingbo Zhou N. Keskar Huan Wang Caiming Xiong LRM SyDa 26 29 0 24 Oct 2020
A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios Michael A. Hedderich Lukas Lange Heike Adel Jannik Strötgen Dietrich Klakow 221 287 0 23 Oct 2020
Continual Learning for Natural Language Generation in Task-oriented Dialog Systems Fei Mi Liangwei Chen Mengjie Zhao Minlie Huang Boi Faltings CLL KELM 19 68 0 02 Oct 2020
Automated Source Code Generation and Auto-completion Using Deep Learning: Comparing and Discussing Current Language-Model-Related Approaches Juan Cruz-Benito Sanjay Vishwakarma Francisco Martín-Fernández Ismael Faro Ibm Quantum 22 30 0 16 Sep 2020
It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners Timo Schick Hinrich Schütze 51 956 0 15 Sep 2020
TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech Andy T. Liu Shang-Wen Li Hung-yi Lee SSL 62 356 0 12 Jul 2020
Learning Spoken Language Representations with Neural Lattice Language Modeling Chao-Wei Huang Yun-Nung Vivian 8 10 0 06 Jul 2020
A Further Study of Unsupervised Pre-training for Transformer Based Speech Recognition Dongwei Jiang Wubo Li Ruixiong Zhang Miao Cao Ne Luo Yang Han Wei Zou Xiangang Li SSL 25 29 0 20 May 2020
Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting Sanyuan Chen Yutai Hou Yiming Cui Wanxiang Che Ting Liu Xiangzhan Yu KELM CLL 21 213 0 27 Apr 2020
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks Suchin Gururangan Ana Marasović Swabha Swayamdipta Kyle Lo Iz Beltagy Doug Downey Noah A. Smith VLM AI4CE CLL 72 2,360 0 23 Apr 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 246 1,454 0 18 Mar 2020
A Survey on Contextual Embeddings Qi Liu Matt J. Kusner Phil Blunsom 225 146 0 16 Mar 2020
Attend to the beginning: A study on using bidirectional attention for extractive summarization Ahmed Magooda C. Marcjan 9 2 0 09 Feb 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,591 0 21 Jan 2020
MedCAT -- Medical Concept Annotation Tool Z. Kraljevic D. Bean Aurelie Mascio Lukasz Roguski A. Folarin A. Roberts R. Bendayan Richard J. B. Dobson MedIm 26 29 0 18 Dec 2019