gzip Predicts Data-dependent Scaling Laws

26 May 2024

Papers citing "gzip Predicts Data-dependent Scaling Laws"

11 / 11 papers shown

Title
BioVFM-21M: Benchmarking and Scaling Self-Supervised Vision Foundation Models for Biomedical Image Analysis Jiarun Liu Hong-Yu Zhou Weijian Huang Hao Yang Dongning Song Tao Tan Yong Liang Shanshan Wang MedIm 26 0 0 14 May 2025
Bayesian scaling laws for in-context learning Aryaman Arora Dan Jurafsky Christopher Potts Noah D. Goodman 24 2 0 21 Oct 2024
A Hitchhiker's Guide to Scaling Law Estimation Leshem Choshen Yang Zhang Jacob Andreas 41 6 0 15 Oct 2024
CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models Jiawei Gu Zacc Yang Chuanghao Ding Rui Zhao Fei Tan CLL 42 4 0 24 Jul 2024
Entropy Law: The Story Behind Data Compression and LLM Performance Mingjia Yin Chuhan Wu Yufei Wang Hao Wang Wei Guo Yasheng Wang Y. Liu Ruiming Tang Defu Lian Enhong Chen 42 19 0 09 Jul 2024
RegMix: Data Mixture as Regression for Language Model Pre-training Qian Liu Xiaosen Zheng Niklas Muennighoff Guangtao Zeng Longxu Dou Tianyu Pang Jing Jiang Min-Bin Lin MoE 71 40 1 01 Jul 2024
Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations Alexander Hägele Elie Bakouch Atli Kosson Loubna Ben Allal Leandro von Werra Martin Jaggi 38 34 0 28 May 2024
Compression Represents Intelligence Linearly Yuzhen Huang Jinghan Zhang Zifei Shan Junxian He 50 26 0 15 Apr 2024
Mission: Impossible Language Models Julie Kallini Isabel Papadimitriou Richard Futrell Kyle Mahowald Christopher Potts ELM LRM 44 19 0 12 Jan 2024
DeepSeek LLM: Scaling Open-Source Language Models with Longtermism DeepSeek-AI Xiao Bi : Xiao Bi Deli Chen Guanting Chen ... Yao Zhao Shangyan Zhou Shunfeng Zhou Qihao Zhu Yuheng Zou LRM ALM 139 306 0 05 Jan 2024
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 243 4,469 0 23 Jan 2020