Self-training Improves Pre-training for Natural Language Understanding

5 October 2020

Papers citing "Self-training Improves Pre-training for Natural Language Understanding"

44 / 44 papers shown

Title
REFINE-AF: A Task-Agnostic Framework to Align Language Models via Self-Generated Instructions using Reinforcement Learning from Automated Feedback Aniruddha Roy Pretam Ray Abhilash Nandy Somak Aditya Pawan Goyal ALM 34 0 0 10 May 2025
SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domains Ran Xu Hui Liu Sreyashi Nag Zhenwei Dai Yaochen Xie ... Chen Luo Yang Li Joyce C. Ho Carl Yang Qi He RALM 78 8 0 28 Jan 2025
Understanding Layer Significance in LLM Alignment Guangyuan Shi Zexin Lu Xiaoyu Dong Wenlong Zhang Xuanyu Zhang Yujie Feng Xiao-Ming Wu 58 2 0 23 Oct 2024
A Self-enhancement Multitask Framework for Unsupervised Aspect Category Detection Thi-Nhung Nguyen Hoang Ngo Kiem-Hieu Nguyen Tuan-Dung Cao 29 0 0 16 Nov 2023
How Well Do Text Embedding Models Understand Syntax? Yan Zhang Zhaopeng Feng Zhiyang Teng Zuozhu Liu Haizhou Li 42 3 0 14 Nov 2023
Zero-Shot End-to-End Spoken Language Understanding via Cross-Modal Selective Self-Training Jianfeng He Julian Salazar Kaisheng Yao Haoqi Li Jason (Jinglun) Cai VLM 17 7 0 22 May 2023
Discovering Language Model Behaviors with Model-Written Evaluations Ethan Perez Sam Ringer Kamilė Lukošiūtė Karina Nguyen Edwin Chen ... Danny Hernandez Deep Ganguli Evan Hubinger Nicholas Schiefer Jared Kaplan ALM 22 367 0 19 Dec 2022
DuNST: Dual Noisy Self Training for Semi-Supervised Controllable Text Generation Yuxi Feng Xiaoyuan Yi Xiting Wang L. Lakshmanan Xing Xie DiffM 35 5 0 16 Dec 2022
Self-Transriber: Few-shot Lyrics Transcription with Self-training Xiaoxue Gao Xianghu Yue Haizhou Li 30 7 0 18 Nov 2022
Self-Training with Purpose Preserving Augmentation Improves Few-shot Generative Dialogue State Tracking Jihyun Lee C. Lee Yunsu Kim G. G. Lee 22 0 0 17 Nov 2022
Zero-Shot Text Classification with Self-Training Ariel Gera Alon Halfon Eyal Shnarch Yotam Perlitz L. Ein-Dor Noam Slonim VLM 31 59 0 31 Oct 2022
The State of Profanity Obfuscation in Natural Language Processing Debora Nozza Dirk Hovy 42 7 0 14 Oct 2022
Learning functional sections in medical conversations: iterative pseudo-labeling and human-in-the-loop approach Mengqian Wang Ilya Valmianski X. Amatriain Anitha Kannan 31 2 0 06 Oct 2022
Pseudo-Labels Are All You Need Bogdan Kostić Mathis Lucka Julian Risch 29 2 0 19 Aug 2022
Conformal Credal Self-Supervised Learning Julian Lienen Caglar Demir Eyke Hüllermeier 29 13 0 30 May 2022
AANG: Automating Auxiliary Learning Lucio Dery Paul Michel M. Khodak Graham Neubig Ameet Talwalkar 41 9 0 27 May 2022
Leveraging QA Datasets to Improve Generative Data Augmentation Dheeraj Mekala Tu Vu Timo Schick Jingbo Shang 27 18 0 25 May 2022
The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training Gi-Cheon Kang Sungdong Kim Jin-Hwa Kim Donghyun Kwak Byoung-Tak Zhang 32 10 0 25 May 2022
Learning Action Conditions from Instructional Manuals for Instruction Understanding Te-Lin Wu Caiqi Zhang Qingyuan Hu Alexander Spangher Nanyun Peng 29 4 0 25 May 2022
Few-shot Mining of Naturally Occurring Inputs and Outputs Mandar Joshi Terra Blevins M. Lewis Daniel S. Weld Luke Zettlemoyer 33 1 0 09 May 2022
Improving In-Context Few-Shot Learning via Self-Supervised Training Mingda Chen Jingfei Du Ramakanth Pasunuru Todor Mihaylov Srini Iyer Ves Stoyanov Zornitsa Kozareva SSL AI4MH 38 64 0 03 May 2022
Protecting Intellectual Property of Language Generation APIs with Lexical Watermark Xuanli He Qiongkai Xu Lingjuan Lyu Fangzhao Wu Chenguang Wang WaLM 177 95 0 05 Dec 2021
VarCLR: Variable Semantic Representation Pre-training via Contrastive Learning Qibin Chen Jeremy Lacomis Edward J. Schwartz Graham Neubig Bogdan Vasilescu Claire Le Goues VLM 24 34 0 05 Dec 2021
LOGEN: Few-shot Logical Knowledge-Conditioned Text Generation with Self-training Shumin Deng Jiacheng Yang Hongbin Ye Chuanqi Tan Mosha Chen Songfang Huang Fei Huang Huajun Chen Ningyu Zhang 27 7 0 02 Dec 2021
Diverse Distributions of Self-Supervised Tasks for Meta-Learning in NLP Trapit Bansal K. Gunasekaran Tong Wang Tsendsuren Munkhdalai Andrew McCallum SSL OOD 51 19 0 02 Nov 2021
Self-Supervised Representation Learning: Introduction, Advances and Challenges Linus Ericsson Henry Gouk Chen Change Loy Timothy M. Hospedales SSL OOD AI4TS 34 273 0 18 Oct 2021
Data Augmentation Approaches in Natural Language Processing: A Survey Bohan Li Yutai Hou Wanxiang Che 132 274 0 05 Oct 2021
Self-Training with Differentiable Teacher Simiao Zuo Yue Yu Chen Liang Haoming Jiang Siawpeng Er Chao Zhang T. Zhao H. Zha 46 14 0 15 Sep 2021
Task-adaptive Pre-training and Self-training are Complementary for Natural Language Understanding Shiyang Li Semih Yavuz Wenhu Chen Xifeng Yan 22 12 0 14 Sep 2021
STraTA: Self-Training with Task Augmentation for Better Few-shot Learning Tu Vu Minh-Thang Luong Quoc V. Le Grady Simon Mohit Iyyer 131 61 0 13 Sep 2021
AutoTriggER: Label-Efficient and Robust Named Entity Recognition with Auxiliary Trigger Extraction Dong-Ho Lee Ravi Kiran Selvam Sheikh Muhammad Sarwar Bill Yuchen Lin Fred Morstatter Jay Pujara Elizabeth Boschee James Allan Xiang Ren 31 2 0 10 Sep 2021
Nearest Neighbour Few-Shot Learning for Cross-lingual Classification M Saiful Bari Batool Haider Saab Mansour VLM 19 13 0 06 Sep 2021
Self-training Improves Pre-training for Few-shot Learning in Task-oriented Dialog Systems Fei Mi Wanhao Zhou Feng Cai Lingjing Kong Minlie Huang Boi Faltings 27 32 0 28 Aug 2021
Multi-Task Self-Training for Learning General Representations Golnaz Ghiasi Barret Zoph E. D. Cubuk Quoc V. Le Nayeon Lee SSL 24 100 0 25 Aug 2021
Improved Text Classification via Contrastive Adversarial Training Lin Pan Chung-Wei Hang Avirup Sil Saloni Potdar AAML 28 86 0 21 Jul 2021
Named Entity Recognition with Small Strongly Labeled and Large Weakly Labeled Data Haoming Jiang Danqing Zhang Tianyu Cao Bing Yin T. Zhao NoLa 30 44 0 16 Jun 2021
Generate, Annotate, and Learn: NLP with Synthetic Text Xuanli He Islam Nassar J. Kiros Gholamreza Haffari Mohammad Norouzi 39 51 0 11 Jun 2021
XtremeDistilTransformers: Task Transfer for Task-agnostic Distillation Subhabrata Mukherjee Ahmed Hassan Awadallah Jianfeng Gao 19 22 0 08 Jun 2021
Improving Cross-Lingual Reading Comprehension with Self-Training Wei Huang Chien-yu Huang Hung-yi Lee LRM 36 1 0 08 May 2021
Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation Max Bartolo Tristan Thrush Robin Jia Sebastian Riedel Pontus Stenetorp Douwe Kiela AAML 28 103 0 18 Apr 2021
Crossing the Conversational Chasm: A Primer on Natural Language Processing for Multilingual Task-Oriented Dialogue Systems E. Razumovskaia Goran Glavavs Olga Majewska Edoardo Ponti Anna Korhonen Ivan Vulić 30 32 0 17 Apr 2021
Large-Scale Self- and Semi-Supervised Learning for Speech Translation Changhan Wang Anne Wu J. Pino Alexei Baevski Michael Auli Alexis Conneau SSL 31 44 0 14 Apr 2021
Cycle Self-Training for Domain Adaptation Hong Liu Jianmin Wang Mingsheng Long 38 174 0 05 Mar 2021
Revisiting Self-Training for Neural Sequence Generation Junxian He Jiatao Gu Jiajun Shen MarcÁurelio Ranzato SSL LRM 244 269 0 30 Sep 2019