HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy

26 January 2024

Shi Feng

Papers citing "HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy"

23 / 23 papers shown

Title
YUAN 2.0: A Large Language Model with Localized Filtering-based Attention Shaohua Wu Xudong Zhao Shenling Wang Jiangang Luo Lingjun Li ... Wei Wang Tong Yu Rongguo Zhang Jiahua Zhang Chao Wang OSLM 70 6 0 27 Nov 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 180 4,085 0 09 Jun 2023
Fine-Tuning Language Models with Just Forward Passes Sadhika Malladi Tianyu Gao Eshaan Nichani Alexandru Damian Jason D. Lee Danqi Chen Sanjeev Arora 88 182 0 27 May 2023
Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning Vladislav Lialin Vijeta Deshpande Anna Rumshisky 50 170 0 28 Mar 2023
Efficient Fine-Tuning of BERT Models on the Edge Danilo Vucetic Mohammadreza Tayaranian M. Ziaeefard J. Clark B. Meyer W. Gross 52 34 0 03 May 2022
Staged Training for Transformer Language Models Sheng Shen Pete Walsh Kurt Keutzer Jesse Dodge Matthew E. Peters Iz Beltagy 18 35 0 11 Mar 2022
Composable Sparse Fine-Tuning for Cross-Lingual Transfer Alan Ansell Edoardo Ponti Anna Korhonen Ivan Vulić CLL MoE 89 137 0 14 Oct 2021
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models Elad Ben-Zaken Shauli Ravfogel Yoav Goldberg 121 1,191 0 18 Jun 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 420 3,952 0 18 Apr 2021
Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM Deepak Narayanan Mohammad Shoeybi Jared Casper P. LeGresley M. Patwary ... Prethvi Kashinkunti J. Bernauer Bryan Catanzaro Amar Phanishayee Matei A. Zaharia MoE 54 667 0 09 Apr 2021
Prefix-Tuning: Optimizing Continuous Prompts for Generation Xiang Lisa Li Percy Liang 142 4,167 0 01 Jan 2021
Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning Zhaojiang Lin Andrea Madotto Pascale Fung 72 159 0 08 Apr 2020
ViGGO: A Video Game Corpus for Data-To-Text Generation in Open-Domain Conversation Juraj Juraska Kevin K. Bowden M. Walker 23 44 0 26 Oct 2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Mohammad Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 279 1,861 0 17 Sep 2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy M. Lewis Luke Zettlemoyer Veselin Stoyanov AIMat 341 24,160 0 26 Jul 2019
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions Christopher Clark Kenton Lee Ming-Wei Chang Tom Kwiatkowski Michael Collins Kristina Toutanova 133 1,475 0 24 May 2019
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems Alex Jinpeng Wang Yada Pruksachatkun Nikita Nangia Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 148 2,287 0 02 May 2019
Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task Tao Yu Rui Zhang Kai-Chou Yang Michihiro Yasunaga Dongxu Wang ... Irene Li Qingning Yao Shanelle Roman Zilin Zhang Dragomir R. Radev RALM 46 1,203 0 24 Sep 2018
WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations Mohammad Taher Pilehvar Jose Camacho-Collados 71 478 0 28 Aug 2018
Know What You Don't Know: Unanswerable Questions for SQuAD Pranav Rajpurkar Robin Jia Percy Liang RALM ELM 147 2,818 0 11 Jun 2018
Neural Network Acceptability Judgments Alex Warstadt Amanpreet Singh Samuel R. Bowman 137 1,390 0 31 May 2018
A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference Adina Williams Nikita Nangia Samuel R. Bowman 332 4,444 0 18 Apr 2017
SQuAD: 100,000+ Questions for Machine Comprehension of Text Pranav Rajpurkar Jian Zhang Konstantin Lopyrev Percy Liang RALM 109 8,067 0 16 Jun 2016