Bag of Tricks for Effective Language Model Pretraining and Downstream
Adaptation: A Case Study on GLUE

Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE

18 February 2023

Liang Ding

Bo Du

Li Shen

Yibing Zhan

Papers citing "Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE"

8 / 8 papers shown

Title
Take Care of Your Prompt Bias! Investigating and Mitigating Prompt Bias in Factual Knowledge Extraction Ziyang Xu Keqin Peng Liang Ding Dacheng Tao Xiliang Lu 34 10 0 15 Mar 2024
Diversifying the Mixture-of-Experts Representation for Language Models with Orthogonal Optimizer Boan Liu Liang Ding Li Shen Keqin Peng Yu Cao Dazhao Cheng Dacheng Tao MoE 36 7 0 15 Oct 2023
Revisiting Token Dropping Strategy in Efficient BERT Pretraining Qihuang Zhong Liang Ding Juhua Liu Xuebo Liu Min Zhang Bo Du Dacheng Tao VLM 29 9 0 24 May 2023
Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao AI4MH 52 238 0 19 Feb 2023
PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao VLM CLL 29 41 0 22 Aug 2022
E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation Qihuang Zhong Liang Ding Juhua Liu Bo Du Dacheng Tao 36 27 0 30 May 2022
Interpretable Proof Generation via Iterative Backward Reasoning Hanhao Qu Yu Cao Jun Gao Liang Ding Ruifeng Xu LRM 134 15 0 22 May 2022
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,956 0 20 Apr 2018