Title
Generated Data with Fake Privacy: Hidden Dangers of Fine-tuning Large Language Models on Generated Data Atilla Akkus Mingjie Li Junjie Chu Junjie Chu Michael Backes Sinem Sav Sinem Sav SILM SyDa 48 1 0 12 Sep 2024
Retro-li: Small-Scale Retrieval Augmented Generation Supporting Noisy Similarity Searches and Domain Shift Generalization Gentiana Rashiti G. Karunaratne Mrinmaya Sachan Abu Sebastian Abbas Rahimi RALM 39 0 0 12 Sep 2024
AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledge Han Wang Archiki Prasad Elias Stengel-Eskin Joey Tianyi Zhou 82 5 0 11 Sep 2024
Accelerating Large Language Model Pretraining via LFR Pedagogy: Learn, Focus, and Review Neha Prakriya Jui-Nan Yen Cho-Jui Hsieh Jason Cong KELM AI4CE LRM 31 1 0 10 Sep 2024
What is the Role of Small Models in the LLM Era: A Survey Lihu Chen Gaël Varoquaux ALM 63 23 0 10 Sep 2024
Improving Pretraining Data Using Perplexity Correlations Tristan Thrush Christopher Potts Tatsunori Hashimoto 32 17 0 09 Sep 2024
Residual Stream Analysis with Multi-Layer SAEs Tim Lawson Lucy Farnik Conor Houghton Laurence Aitchison 26 3 0 06 Sep 2024
How Does Code Pretraining Affect Language Model Task Performance? Jackson Petty Sjoerd van Steenkiste Tal Linzen 65 8 0 06 Sep 2024
Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decoding Cheng Wang Yiwei Wang Bryan Hooi Yujun Cai Nanyun Peng Kai-Wei Chang 42 2 0 05 Sep 2024
The Compressor-Retriever Architecture for Language Model OS Yuan Yang Siheng Xiong Ehsan Shareghi Faramarz Fekri RALM KELM 32 1 0 02 Sep 2024
Breaking Class Barriers: Efficient Dataset Distillation via Inter-Class Feature Compensator Xin Zhang Jiawei Du Ping Liu Joey Tianyi Zhou DD 50 2 0 13 Aug 2024
Why Are My Prompts Leaked? Unraveling Prompt Extraction Threats in Customized Large Language Models Zi Liang Haibo Hu Qingqing Ye Yaxin Xiao Haoyang Li AAML ELM SILM 48 6 0 05 Aug 2024
Tamper-Resistant Safeguards for Open-Weight LLMs Rishub Tamirisa Bhrugu Bharathi Long Phan Andy Zhou Alice Gatti ... Andy Zou Dawn Song Bo Li Dan Hendrycks Mantas Mazeika AAML MU 51 41 0 01 Aug 2024
MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning Yupeng Chen Senmiao Wang Zhihang Lin Zhihang Lin Yushun Zhang Tian Ding Ruoyu Sun Ruoyu Sun CLL 80 1 0 30 Jul 2024
Learn while Unlearn: An Iterative Unlearning Framework for Generative Language Models Haoyu Tang Ye Liu Xukai Liu Xukai Liu Yanghai Zhang Kai Zhang Xiaofang Zhou Enhong Chen MU 75 3 0 25 Jul 2024
Compensate Quantization Errors+: Quantized Models Are Inquisitive Learners Yifei Gao Jie Ou Lei Wang Fanhua Shang Jaji Wu MQ 47 0 0 22 Jul 2024
Beyond Next Token Prediction: Patch-Level Training for Large Language Models Chenze Shao Fandong Meng Jie Zhou 49 1 0 17 Jul 2024
Genomic Language Models: Opportunities and Challenges Gonzalo Benegas Chengzhong Ye C. Albors Jianan Canal Li Yun S. Song AI4CE LM&MA ELM 43 18 0 16 Jul 2024
Training on the Test Task Confounds Evaluation and Emergence Ricardo Dominguez-Olmedo Florian E. Dorner Moritz Hardt ELM 71 7 1 10 Jul 2024
How Effective are State Space Models for Machine Translation? Hugo Pitorro Pavlo Vasylenko Marcos Vinícius Treviso André F. T. Martins Mamba 45 2 0 07 Jul 2024
IL-TUR: Benchmark for Indian Legal Text Understanding and Reasoning Abhinav Joshi Shounak Paul Akshat Sharma Pawan Goyal Saptarshi Ghosh Ashutosh Modi AILaw ELM 34 7 0 07 Jul 2024
From 'Showgirls' to 'Performers': Fine-tuning with Gender-inclusive Language for Bias Reduction in LLMs Marion Bartl Susan Leavy 43 8 0 05 Jul 2024
Learning to (Learn at Test Time): RNNs with Expressive Hidden States Yu Sun Xinhao Li Karan Dalal Jiarui Xu Arjun Vikram ... Xinlei Chen Xiaolong Wang Sanmi Koyejo Tatsunori Hashimoto Carlos Guestrin 63 92 0 05 Jul 2024
Normalization and effective learning rates in reinforcement learning Clare Lyle Zeyu Zheng Khimya Khetarpal James Martens H. V. Hasselt Razvan Pascanu Will Dabney 19 7 0 01 Jul 2024
RegMix: Data Mixture as Regression for Language Model Pre-training Qian Liu Xiaosen Zheng Niklas Muennighoff Guangtao Zeng Longxu Dou Tianyu Pang Jing Jiang Min-Bin Lin MoE 74 40 1 01 Jul 2024
Look Ahead or Look Around? A Theoretical Comparison Between Autoregressive and Masked Pretraining Qi Zhang Tianqi Du Haotian Huang Yifei Wang Yisen Wang 39 3 0 01 Jul 2024
Resolving Discrepancies in Compute-Optimal Scaling of Language Models Tomer Porian Mitchell Wortsman J. Jitsev Ludwig Schmidt Y. Carmon 60 20 0 27 Jun 2024
Improving Robustness of LLM-based Speech Synthesis by Learning Monotonic Alignment Paarth Neekhara Shehzeen Samarah Hussain Subhankar Ghosh Jason Chun Lok Li Rafael Valle Rohan Badlani Boris Ginsburg 55 11 0 25 Jun 2024
Brittle Minds, Fixable Activations: Understanding Belief Representations in Language Models Matteo Bortoletto Constantin Ruhdorfer Lei Shi Andreas Bulling AI4MH LRM 46 5 0 25 Jun 2024
Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon USVSN Sai Prashanth Alvin Deng Kyle O'Brien Jyothir S V Mohammad Aflah Khan ... Jacob Ray Fuehne Stella Biderman Tracy Ke Katherine Lee Naomi Saphra 63 12 0 25 Jun 2024
Blind Baselines Beat Membership Inference Attacks for Foundation Models Debeshee Das Jie Zhang Florian Tramèr MIALM 85 28 1 23 Jun 2024
DeciMamba: Exploring the Length Extrapolation Potential of Mamba Assaf Ben-Kish Itamar Zimerman Shady Abu Hussein Nadav Cohen Amir Globerson Lior Wolf Raja Giryes Mamba 77 13 0 20 Jun 2024
Fantastic Copyrighted Beasts and How (Not) to Generate Them Luxi He Yangsibo Huang Weijia Shi Tinghao Xie Haotian Liu Yue Wang Luke Zettlemoyer Chiyuan Zhang Danqi Chen Peter Henderson 46 9 0 20 Jun 2024
Design and evaluation of AI copilots -- case studies of retail copilot templates Michal Furmakiewicz Chang Liu Angus Taylor Ilya Venger 34 2 0 17 Jun 2024
ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking Wenshuo Li Xinghao Chen Han Shu Yehui Tang Yunhe Wang MQ 45 2 0 17 Jun 2024
Data Shapley in One Training Run Jiachen T. Wang Prateek Mittal Dawn Song Ruoxi Jia TDI 42 7 0 16 Jun 2024
Multilingual Large Language Models and Curse of Multilinguality Daniil Gurgurov Tanja Bäumel Tatiana Anikina 86 4 0 15 Jun 2024
REVS: Unlearning Sensitive Information in Language Models via Rank Editing in the Vocabulary Space Tomer Ashuach Martin Tutek Yonatan Belinkov KELM MU 71 4 0 13 Jun 2024
Can I understand what I create? Self-Knowledge Evaluation of Large Language Models Zhiquan Tan Lai Wei Jindong Wang Xing Xie Weiran Huang ELM LRM 35 5 0 10 Jun 2024
Hidden Holes: topological aspects of language models Stephen Fitz P. Romero Jiyan Jonas Schneider 35 0 0 09 Jun 2024
LCQ: Low-Rank Codebook based Quantization for Large Language Models Wen-Pu Cai Wu-Jun Li Wu-Jun Li MQ 46 0 0 31 May 2024
Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models Zachary Ankner Cody Blakeney Kartik K. Sreenivasan Max Marion Matthew L. Leavitt Mansheej Paul 43 24 0 30 May 2024
A Survey Study on the State of the Art of Programming Exercise Generation using Large Language Models Eduard Frankford Ingo Höhn Clemens Sauerwein Ruth Breu ELM 39 2 0 30 May 2024
Linguistic Collapse: Neural Collapse in (Large) Language Models Robert Wu V. Papyan 48 12 0 28 May 2024
Glauber Generative Model: Discrete Diffusion Models via Binary Classification Harshit Varma Dheeraj M. Nagaraj Karthikeyan Shanmugam VLM 64 2 0 27 May 2024
Scaling Laws for Discriminative Classification in Large Language Models Dean Wyatte Fatemeh Tahmasbi Ming Li Thomas Markovich 44 2 0 24 May 2024
The Mosaic Memory of Large Language Models Igor Shilov Matthieu Meeus Yves-Alexandre de Montjoye 47 3 0 24 May 2024
Emergence of a High-Dimensional Abstraction Phase in Language Transformers Emily Cheng Diego Doimo Corentin Kervadec Iuri Macocco Jade Yu A. Laio Marco Baroni 112 11 0 24 May 2024
Proving Theorems Recursively Haiming Wang Huajian Xin Zhengying Liu Wenda Li Yinya Huang ... Zhicheng YANG Jing Tang Jian Yin Zhenguo Li Xiaodan Liang LRM 41 12 0 23 May 2024
A Multi-Perspective Analysis of Memorization in Large Language Models Bowen Chen Namgi Han Yusuke Miyao 46 1 0 19 May 2024