To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks

14 March 2019

Papers citing "To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks"

50 / 229 papers shown

Title
Training Neural Networks with Fixed Sparse Masks Yi-Lin Sung Varun Nair Colin Raffel FedML 32 197 0 18 Nov 2021
Scalable Diverse Model Selection for Accessible Transfer Learning Daniel Bolya Rohit Mittapalli Judy Hoffman OODD 27 41 0 12 Nov 2021
Multilingual and Multilabel Emotion Recognition using Virtual Adversarial Training Vikram Gupta 26 4 0 11 Nov 2021
Focusing on Potential Named Entities During Active Label Acquisition Ali Osman Berk Şapcı Oznur Tastan Reyyan Yeniterzi 27 2 0 06 Nov 2021
Control Prefixes for Parameter-Efficient Text Generation Jordan Clive Kris Cao Marek Rei 44 32 0 15 Oct 2021
Learning Natural Language Generation from Scratch Alice Martin Donati Guillaume Quispe Charles Ollion Sylvain Le Corff Florian Strub Olivier Pietquin LRM 26 4 0 20 Sep 2021
Navigating the Kaleidoscope of COVID-19 Misinformation Using Deep Learning Yuanzhi Chen Mohammad Rashedul Hasan 24 4 0 19 Sep 2021
On the Universality of Deep Contextual Language Models Shaily Bhatt Poonam Goyal Sandipan Dandapat Monojit Choudhury Sunayana Sitaram ELM 25 5 0 15 Sep 2021
A Conditional Generative Matching Model for Multi-lingual Reply Suggestion Budhaditya Deb Guoqing Zheng Milad Shokouhi Ahmed Hassan Awadallah 36 1 0 15 Sep 2021
The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders Han He Jinho Choi 51 87 0 14 Sep 2021
How Does Fine-tuning Affect the Geometry of Embedding Space: A Case Study on Isotropy S. Rajaee Mohammad Taher Pilehvar 79 20 0 10 Sep 2021
Sense representations for Portuguese: experiments with sense embeddings and deep neural language models Jéssica Rodrigues da Silva Helena de Medeiros Caseli 13 3 0 31 Aug 2021
Fine-Tuning Pretrained Language Models With Label Attention for Biomedical Text Classification Bruce Nguyen Shaoxiong Ji MedIm 11 3 0 26 Aug 2021
Towards Zero-shot Language Modeling Edoardo Ponti Ivan Vulić Ryan Cotterell Roi Reichart Anna Korhonen 30 19 0 06 Aug 2021
Robust Transfer Learning with Pretrained Language Models through Adapters Wenjuan Han Bo Pang Ying Nian Wu 16 54 0 05 Aug 2021
Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi Graham Neubig VLM SyDa 61 3,838 0 28 Jul 2021
A Flexible Multi-Task Model for BERT Serving Tianwen Wei Jianwei Qi Shenghuang He 34 7 0 12 Jul 2021
A Theoretical Analysis of Fine-tuning with Linear Teachers Gal Shachaf Alon Brutzkus Amir Globerson 34 17 0 04 Jul 2021
What do End-to-End Speech Models Learn about Speaker, Language and Channel Information? A Layer-wise and Neuron-level Analysis Shammur A. Chowdhury Nadir Durrani Ahmed M. Ali 41 12 0 01 Jul 2021
Cross-lingual alignments of ELMo contextual embeddings Matej Ulčar Marko Robnik-Šikonja 24 17 0 30 Jun 2021
A Closer Look at How Fine-tuning Changes BERT Yichu Zhou Vivek Srikumar 26 63 0 27 Jun 2021
An Empirical Study on Hyperparameter Optimization for Fine-Tuning Pre-trained Language Models Xueqing Liu Chi Wang 17 18 0 17 Jun 2021
Why Can You Lay Off Heads? Investigating How BERT Heads Transfer Ting-Rui Chiang Yun-Nung Chen 36 0 0 14 Jun 2021
Robust Knowledge Graph Completion with Stacked Convolutions and a Student Re-Ranking Network Justin Lovelace Denis R. Newman-Griffis Shikhar Vashishth J. Lehman Carolyn Rose OffRL 28 31 0 11 Jun 2021
Neural Supervised Domain Adaptation by Augmenting Pre-trained Models with Random Units Sara Meftah N. Semmar Y. Tamaazousti H. Essafi F. Sadat 20 3 0 09 Jun 2021
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers Rabeeh Karimi Mahabadi James Henderson Sebastian Ruder MoE 67 468 0 08 Jun 2021
Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks Rabeeh Karimi Mahabadi Sebastian Ruder Mostafa Dehghani James Henderson MoE 39 294 0 08 Jun 2021
Annotation Curricula to Implicitly Train Non-Expert Annotators Ji-Ung Lee Jan-Christoph Klie Iryna Gurevych 16 11 0 04 Jun 2021
Language Scaling for Universal Suggested Replies Model Qianlan Ying Payal Bajaj Budhaditya Deb Yu Yang Wei Wang Bojia Lin Milad Shokouhi Xia Song Yang Yang Daxin Jiang LRM 13 2 0 04 Jun 2021
The Low-Dimensional Linear Geometry of Contextualized Word Representations Evan Hernandez Jacob Andreas MILM 28 40 0 15 May 2021
Adapting by Pruning: A Case Study on BERT Yang Gao Nicolo Colombo Wen Wang 21 17 0 07 May 2021
Morph Call: Probing Morphosyntactic Content of Multilingual Transformers Vladislav Mikhailov O. Serikov Ekaterina Artemova 12 9 0 26 Apr 2021
skweak: Weak Supervision Made Easy for NLP Pierre Lison Jeremy Barnes A. Hubin 27 43 0 19 Apr 2021
Effect of Post-processing on Contextualized Word Representations Hassan Sajjad Firoj Alam Fahim Dalvi Nadir Durrani 6 9 0 15 Apr 2021
Large-Scale Contextualised Language Modelling for Norwegian Andrey Kutuzov Jeremy Barnes Erik Velldal Lilja Ovrelid Stephan Oepen 27 38 0 13 Apr 2021
DirectProbe: Studying Representations without Classifiers Yichu Zhou Vivek Srikumar 32 27 0 13 Apr 2021
Fine-Tuning Transformers for Identifying Self-Reporting Potential Cases and Symptoms of COVID-19 in Tweets Max Fleming Priyanka Dondeti C. Dreisbach Adam Poliak 14 2 0 12 Apr 2021
Affinity-Based Hierarchical Learning of Dependent Concepts for Human Activity Recognition A. Osmani Massinissa Hamidi Pegah Alizadeh 20 1 0 11 Apr 2021
What's the best place for an AI conference, Vancouver or ______: Why completing comparative questions is difficult Avishai Zagoury Einat Minkov Idan Szpektor William W. Cohen ELM 35 6 0 05 Apr 2021
Scaling Creative Inspiration with Fine-Grained Functional Aspects of Ideas Tom Hope Ronen Tamari Hyeonsu B Kang Daniel Hershcovich Joel Chan A. Kittur Dafna Shahaf 14 18 0 19 Feb 2021
Customizing Contextualized Language Models forLegal Document Reviews Shohreh Shaghaghian Luna Feng Feng Borna Jafarpour Nicolai Pogrebnyakov AILaw 22 19 0 10 Feb 2021
CSS-LM: A Contrastive Framework for Semi-supervised Fine-tuning of Pre-trained Language Models Yusheng Su Xu Han Yankai Lin Zhengyan Zhang Zhiyuan Liu Peng Li Jie Zhou Maosong Sun 19 10 0 07 Feb 2021
On the Evolution of Syntactic Information Encoded by BERT's Contextualized Representations Laura Pérez-Mayos Roberto Carlini Miguel Ballesteros Leo Wanner 29 7 0 27 Jan 2021
Situation and Behavior Understanding by Trope Detection on Films Chen-Hsi Chang Hung-Ting Su Jui-Heng Hsu Yu-Siang Wang Yu-Cheng Chang Zhe-Yu Liu Ya-Liang Chang Wen-Feng Cheng Ke-Jyun Wang Winston H. Hsu 16 6 0 19 Jan 2021
Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing Minh Nguyen Viet Dac Lai Amir Pouran Ben Veyseh Thien Huu Nguyen 52 132 0 09 Jan 2021
WARP: Word-level Adversarial ReProgramming Karen Hambardzumyan Hrant Khachatrian Jonathan May AAML 254 342 0 01 Jan 2021
An Experimental Evaluation of Transformer-based Language Models in the Biomedical Domain Paul Grouchy Shobhit Jain Michael Liu Kuhan Wang Max Tian Nidhi Arora Hillary Ngai Faiza Khan Khattak Elham Dolatabadi S. Kocak LM&MA MedIm 16 4 0 31 Dec 2020
Leveraging GPT-2 for Classifying Spam Reviews with Limited Labeled Data via Adversarial Training Athirai Aravazhi Irissappane Hanfei Yu Yankun Shen Anubha Agrawal Gray Stanton 19 9 0 24 Dec 2020
SChuBERT: Scholarly Document Chunks with BERT-encoding boost Citation Count Prediction Thomas van Dongen Gideon Maillette de Buy Wenniger Lambert Schomaker 24 24 0 21 Dec 2020
BERT Goes Shopping: Comparing Distributional Models for Product Representations Federico Bianchi Bingqing Yu Jacopo Tagliabue 25 15 0 17 Dec 2020