NLP From Scratch Without Large-Scale Pretraining: A Simple and Efficient
Framework

v1v2 (latest)

NLP From Scratch Without Large-Scale Pretraining: A Simple and Efficient Framework

7 November 2021

ArXiv (abs)PDF HTML

Papers citing "NLP From Scratch Without Large-Scale Pretraining: A Simple and Efficient Framework"

13 / 13 papers shown

Title
Mixtraining: A Better Trade-Off Between Compute and Performance Zexin Li Jiancheng Zhang Yufei Li Yinglun Zhu Cong Liu 71 0 0 26 Feb 2025
Generative Deduplication For Socia Media Data Selection Xianming Li Jing Li 91 2 0 11 Jan 2024
GIO: Gradient Information Optimization for Training Dataset Selection Dante Everaert Christopher Potts 118 6 0 20 Jun 2023
Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Model Xiao Wang Wei Zhou Qi Zhang Jie Zhou Songyang Gao Junzhe Wang Menghan Zhang Xiang Gao Yunwen Chen Tao Gui 129 10 0 22 May 2023
VarMAE: Pre-training of Variational Masked Autoencoder for Domain-adaptive Language Understanding Dou Hu Xiaolong Hou Xiyang Du Mengyuan Zhou Lian-Xin Jiang Yang Mo Xiaofeng Shi 97 13 0 01 Nov 2022
Knowledge-grounded Dialog State Tracking Dian Yu Mingqiu Wang Yuan Cao Izhak Shafran Laurent El Shafey H. Soltau BDL 77 3 0 13 Oct 2022
Parameter-Efficient Tuning with Special Token Adaptation Xiaoocong Yang James Y. Huang Wenxuan Zhou Muhao Chen 89 12 0 10 Oct 2022
Downstream Datasets Make Surprisingly Good Pretraining Corpora Kundan Krishna Saurabh Garg Jeffrey P. Bigham Zachary Chase Lipton 108 33 0 28 Sep 2022
Domain-Specific NER via Retrieving Correlated Samples Xin Zhang Yong Jiang Xiaobin Wang Xuming Hu Yueheng Sun Pengjun Xie Meishan Zhang 120 15 0 27 Aug 2022
u-HuBERT: Unified Mixed-Modal Speech Pretraining And Zero-Shot Transfer to Unlabeled Modality Wei-Ning Hsu Bowen Shi SSL VLM 108 43 0 14 Jul 2022
ORCA: Interpreting Prompted Language Models via Locating Supporting Data Evidence in the Ocean of Pretraining Data Xiaochuang Han Yulia Tsvetkov 92 31 0 25 May 2022
AdaPrompt: Adaptive Model Training for Prompt-based NLP Yulong Chen Yang Liu Li Dong Shuohang Wang Chenguang Zhu Michael Zeng Yue Zhang VLM 102 48 0 10 Feb 2022
Should We Be Pre-training? An Argument for End-task Aware Training as an Alternative Lucio Dery Paul Michel Ameet Talwalkar Graham Neubig CLL 90 35 0 15 Sep 2021