ResearchTrend.AI
Faster Convergence for Transformer Fine-tuning with Line Search Methods

Faster Convergence for Transformer Fine-tuning with Line Search Methods

Philip Kenneweg
Leonardo Galli
Tristan Kenneweg
Barbara Hammer
    ODL
ArXivPDFHTML

Papers citing "Faster Convergence for Transformer Fine-tuning with Line Search Methods"

5 / 5 papers shown
Title
No learning rates needed: Introducing SALSA -- Stable Armijo Line Search Adaptation
No learning rates needed: Introducing SALSA -- Stable Armijo Line Search Adaptation
Philip Kenneweg
Tristan Kenneweg
Fabian Fumagalli
Barbara Hammer
ODL
21
0
0
30 Jul 2024
Improving Line Search Methods for Large Scale Neural Network Training
Improving Line Search Methods for Large Scale Neural Network Training
Philip Kenneweg
Tristan Kenneweg
Barbara Hammer
ODL
18
3
0
27 Mar 2024
Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be
Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be
Frederik Kunstner
Jacques Chen
J. Lavington
Mark W. Schmidt
40
67
0
27 Apr 2023
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Alex Jinpeng Wang
Amanpreet Singh
Julian Michael
Felix Hill
Omer Levy
Samuel R. Bowman
ELM
299
6,984
0
20 Apr 2018
Densely Connected Convolutional Networks
Densely Connected Convolutional Networks
Gao Huang
Zhuang Liu
L. V. D. van der Maaten
Kilian Q. Weinberger
PINN
3DV
315
36,381
0
25 Aug 2016