ZO-AdaMU Optimizer: Adapting Perturbation by the Momentum and
Uncertainty in Zeroth-order Optimization

ZO-AdaMU Optimizer: Adapting Perturbation by the Momentum and Uncertainty in Zeroth-order Optimization

23 December 2023

Shuoran Jiang

Papers citing "ZO-AdaMU Optimizer: Adapting Perturbation by the Momentum and Uncertainty in Zeroth-order Optimization"

8 / 8 papers shown

Title
MaZO: Masked Zeroth-Order Optimization for Multi-Task Fine-Tuning of Large Language Models Zhen Zhang Yuqing Yang Kai Zhen Nathan Susanj Athanasios Mouchtaris Siegfried Kunzmann Zheng Zhang 54 0 0 17 Feb 2025
Perturbation-Restrained Sequential Model Editing Junjie Ma Hong Wang Haoyang Xu Zhen-Hua Ling Jia-Chen Gu KELM 59 8 0 27 May 2024
Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer Yanjun Zhao Sizhe Dang Haishan Ye Guang Dai Yi Qian Ivor W.Tsang 66 8 0 23 Feb 2024
An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks Yuxiang Wu Yu Zhao Baotian Hu Pasquale Minervini Pontus Stenetorp Sebastian Riedel RALM KELM 51 43 0 30 Oct 2022
A Kernel-Based View of Language Model Fine-Tuning Sadhika Malladi Alexander Wettig Dingli Yu Danqi Chen Sanjeev Arora VLM 68 60 0 11 Oct 2022
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,919 0 31 Dec 2020
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference Timo Schick Hinrich Schütze 258 1,589 0 21 Jan 2020
Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes Ohad Shamir Tong Zhang 101 570 0 08 Dec 2012