Understanding Pre-training and Fine-tuning from Loss Landscape Perspectives

23 May 2025

Papers citing "Understanding Pre-training and Fine-tuning from Loss Landscape Perspectives"

50 / 60 papers shown

Title
Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval Taiye Chen Zeming Wei Ang Li Yisen Wang AAML 33 1 0 21 May 2025
Fundamental Safety-Capability Trade-offs in Fine-tuning Large Language Models Pin-Yu Chen Han Shen Payel Das Tianyi Chen 61 2 0 24 Mar 2025
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates Kaifeng Lyu Haoyu Zhao Xinran Gu Dingli Yu Anirudh Goyal Sanjeev Arora ALM 86 50 0 20 Jan 2025
SaLoRA: Safety-Alignment Preserved Low-Rank Adaptation Mingjie Li Wai Man Si Michael Backes Yang Zhang Yisen Wang 40 12 0 03 Jan 2025
Unraveling and Mitigating Safety Alignment Degradation of Vision-Language Models Qin Liu Chao Shang Ling Liu Nikolaos Pappas Jie Ma Neha Anna John Srikanth Doss Kadarundalagi Raghuram Doss Lluís Marquez Miguel Ballesteros Yassine Benajiba 63 9 0 11 Oct 2024
Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey Tiansheng Huang Sihao Hu Fatih Ilhan Selim Furkan Tekin Ling Liu AAML 69 32 0 26 Sep 2024
A Comprehensive Survey of LLM Alignment Techniques: RLHF, RLAIF, PPO, DPO and More Zhichao Wang Bin Bi Shiva K. Pentyala Kiran Ramnath Sougata Chaudhuri ... Z. Zhu Xiang-Bo Mao S. Asur Na Na Cheng OffRL 47 48 0 23 Jul 2024
Safety Alignment Should Be Made More Than Just a Few Tokens Deep Xiangyu Qi Ashwinee Panda Kaifeng Lyu Xiao Ma Subhrajit Roy Ahmad Beirami Prateek Mittal Peter Henderson 65 99 0 10 Jun 2024
Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt Zonghao Ying Aishan Liu Tianyuan Zhang Zhengmin Yu Siyuan Liang Xianglong Liu Dacheng Tao AAML 64 32 0 06 Jun 2024
Lazy Safety Alignment for Large Language Models against Harmful Fine-tuning Tiansheng Huang Sihao Hu Fatih Ilhan Selim Furkan Tekin Ling Liu 83 29 0 28 May 2024
Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models Sheng-Hsuan Peng Pin-Yu Chen Matthew Hull Duen Horng Chau 59 26 0 27 May 2024
Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models Chia-Yi Hsu Yu-Lin Tsai Chih-Hsun Lin Pin-Yu Chen Chia-Mu Yu Chun-ying Huang 80 44 0 27 May 2024
No Two Devils Alike: Unveiling Distinct Mechanisms of Fine-tuning Attacks Chak Tou Leong Yi Cheng Kaishuai Xu Jian Wang Hanlin Wang Wenjie Li AAML 85 21 0 25 May 2024
Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks Maksym Andriushchenko Francesco Croce Nicolas Flammarion AAML 106 186 0 02 Apr 2024
The Unreasonable Ineffectiveness of the Deeper Layers Andrey Gromov Kushal Tirumala Hassan Shapourian Paolo Glorioso Daniel A. Roberts 63 93 0 26 Mar 2024
Stealing Part of a Production Language Model Nicholas Carlini Daniel Paleka Krishnamurthy Dvijotham Thomas Steinke Jonathan Hayase ... Arthur Conmy Itay Yona Eric Wallace David Rolnick Florian Tramèr MLAU AAML 37 78 0 11 Mar 2024
ShortGPT: Layers in Large Language Models are More Redundant Than You Expect Xin Men Mingyu Xu Qingyu Zhang Bingning Wang Hongyu Lin Yaojie Lu Xianpei Han Weipeng Chen 60 122 0 06 Mar 2024
On the Duality Between Sharpness-Aware Minimization and Adversarial Training Yihao Zhang Hangzhou He Jingyu Zhu Huanran Chen Yifei Wang Zeming Wei AAML 71 12 0 23 Feb 2024
Vaccine: Perturbation-aware Alignment for Large Language Model Tiansheng Huang Sihao Hu Ling Liu 77 39 0 02 Feb 2024
Safe RLHF: Safe Reinforcement Learning from Human Feedback Josef Dai Xuehai Pan Ruiyang Sun Jiaming Ji Xinbo Xu Mickel Liu Yizhou Wang Yaodong Yang 66 330 0 19 Oct 2023
Jailbreaking Black Box Large Language Models in Twenty Queries Patrick Chao Alexander Robey Yan Sun Hamed Hassani George J. Pappas Eric Wong AAML 63 642 0 12 Oct 2023
Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations Zeming Wei Yifei Wang Ang Li Yichuan Mo Yisen Wang 59 258 0 10 Oct 2023
Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! Xiangyu Qi Yi Zeng Tinghao Xie Pin-Yu Chen Ruoxi Jia Prateek Mittal Peter Henderson SILM 82 571 0 05 Oct 2023
Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models Xianjun Yang Xiao Wang Qi Zhang Linda R. Petzold William Y. Wang Xun Zhao Dahua Lin 42 174 0 04 Oct 2023
How Robust is Google's Bard to Adversarial Image Attacks? Yinpeng Dong Huanran Chen Jiawei Chen Zhengwei Fang Xiaohu Yang Yichi Zhang Yu Tian Hang Su Jun Zhu AAML 47 110 0 21 Sep 2023
Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions Federico Bianchi Mirac Suzgun Giuseppe Attanasio Paul Röttger Dan Jurafsky Tatsunori Hashimoto James Zou ALM LM&MA LRM 43 196 0 14 Sep 2023
Fine-tuning can cripple your foundation model; preserving features may be the solution Jishnu Mukhoti Y. Gal Philip Torr P. Dokania CLL 48 37 0 25 Aug 2023
FLIRT: Feedback Loop In-context Red Teaming Ninareh Mehrabi Palash Goyal Christophe Dupuy Qian Hu Shalini Ghosh R. Zemel Kai-Wei Chang Aram Galstyan Rahul Gupta DiffM 42 59 0 08 Aug 2023
Universal and Transferable Adversarial Attacks on Aligned Language Models Andy Zou Zifan Wang Nicholas Carlini Milad Nasr J. Zico Kolter Matt Fredrikson 126 1,376 0 27 Jul 2023
Jailbroken: How Does LLM Safety Training Fail? Alexander Wei Nika Haghtalab Jacob Steinhardt 144 907 0 05 Jul 2023
Visual Adversarial Examples Jailbreak Aligned Large Language Models Xiangyu Qi Kaixuan Huang Ashwinee Panda Peter Henderson Mengdi Wang Prateek Mittal AAML 39 150 0 22 Jun 2023
A Simple and Effective Pruning Approach for Large Language Models Mingjie Sun Zhuang Liu Anna Bair J. Zico Kolter 97 389 0 20 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 165 4,085 0 09 Jun 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 75 13,788 0 15 Mar 2023
Constitutional AI: Harmlessness from AI Feedback Yuntao Bai Saurav Kadavath Sandipan Kundu Amanda Askell John Kernion ... Dario Amodei Nicholas Joseph Sam McCandlish Tom B. Brown Jared Kaplan SyDa MoMe 123 1,552 0 15 Dec 2022
Mechanistic Mode Connectivity Ekdeep Singh Lubana Eric J. Bigelow Robert P. Dick David M. Krueger Hidenori Tanaka 52 45 0 15 Nov 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 264 326 0 11 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 552 12,525 0 04 Mar 2022
Red Teaming Language Models with Language Models Ethan Perez Saffron Huang Francis Song Trevor Cai Roman Ring John Aslanides Amelia Glaese Nat McAleese G. Irving AAML 27 627 0 07 Feb 2022
Training Verifiers to Solve Math Word Problems K. Cobbe V. Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun ... Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse John Schulman ReLM OffRL LRM 165 4,175 0 27 Oct 2021
Evaluating Large Language Models Trained on Code Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pondé ... Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever Wojciech Zaremba ELM ALM 98 5,328 0 07 Jul 2021
Aligning AI With Shared Human Values Dan Hendrycks Collin Burns Steven Basart Andrew Critch Jingkai Li D. Song Jacob Steinhardt 72 540 0 05 Aug 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 261 41,106 0 28 May 2020
Loss landscapes and optimization in over-parameterized non-linear systems and neural networks Chaoyue Liu Libin Zhu M. Belkin ODL 29 258 0 29 Feb 2020
Linear Mode Connectivity and the Lottery Ticket Hypothesis Jonathan Frankle Gintare Karolina Dziugaite Daniel M. Roy Michael Carbin MoMe 105 611 0 11 Dec 2019
Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifiers Guang-He Lee Yang Yuan Shiyu Chang Tommi Jaakkola AAML 25 123 0 12 Jun 2019
Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers Hadi Salman Greg Yang Jungshian Li Pengchuan Zhang Huan Zhang Ilya P. Razenshteyn Sébastien Bubeck AAML 50 544 0 09 Jun 2019
Certified Adversarial Robustness via Randomized Smoothing Jeremy M. Cohen Elan Rosenfeld J. Zico Kolter AAML 83 2,018 0 08 Feb 2019
Reconciling modern machine learning practice and the bias-variance trade-off M. Belkin Daniel J. Hsu Siyuan Ma Soumik Mandal 120 1,628 0 28 Dec 2018
A Convergence Theory for Deep Learning via Over-Parameterization Zeyuan Allen-Zhu Yuanzhi Li Zhao Song AI4CE ODL 99 1,457 0 09 Nov 2018