Towards Theoretically Inspired Neural Initialization Optimization

Towards Theoretically Inspired Neural Initialization Optimization

12 October 2022

Papers citing "Towards Theoretically Inspired Neural Initialization Optimization"

10 / 10 papers shown

Title
Initialization of Large Language Models via Reparameterization to Mitigate Loss Spikes Kosuke Nishida Kyosuke Nishida Kuniko Saito 36 1 0 07 Oct 2024
Advancing Neural Network Performance through Emergence-Promoting Initialization Scheme Johnny Jingze Li V. George Gabriel A. Silva ODL 44 0 0 26 Jul 2024
Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation Yibo Yang Xiaojie Li Motasem Alfarra Hasan Hammoud Adel Bibi Philip Torr Guohao Li 37 2 0 07 Jun 2024
CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning Yibo Yang Xiaojie Li Zhongzhu Zhou Shuaiwen Leon Song Jianlong Wu Liqiang Nie Guohao Li 45 6 0 07 Jun 2024
LoGAH: Predicting 774-Million-Parameter Transformers using Graph HyperNetworks with 1/100 Parameters Xinyu Zhou Boris Knyazev Alexia Jolicoeur-Martineau Jie Fu AI4CE 45 0 0 25 May 2024
GenView: Enhancing View Quality with Pretrained Generative Model for Self-Supervised Learning Xiaojie Li Yibo Yang Hefei Ling Jianlong Wu Yue Yu Guohao Li Min Zhang SSL 34 6 0 18 Mar 2024
Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models? Boris Knyazev Doha Hwang Simon Lacoste-Julien AI4CE 37 17 0 07 Mar 2023
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 950 20,599 0 17 Apr 2017
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 271 5,330 0 05 Nov 2016
Densely Connected Convolutional Networks Gao Huang Zhuang Liu L. V. D. van der Maaten Kilian Q. Weinberger PINN 3DV 315 36,420 0 25 Aug 2016