Compression Represents Intelligence Linearly

Compression Represents Intelligence Linearly

15 April 2024

Papers citing "Compression Represents Intelligence Linearly"

12 / 12 papers shown

Title
Memorization-Compression Cycles Improve Generalization Fangyuan Yu 39 0 0 13 May 2025
Revisiting Transformers through the Lens of Low Entropy and Dynamic Sparsity Ruifeng Ren Yong Liu 126 0 0 26 Apr 2025
Predictive Data Selection: The Data That Predicts Is the Data That Teaches Kashun Shum Y. Huang Hongjian Zou Qi Ding Yixuan Liao X. Chen Qian Liu Junxian He 67 2 0 02 Mar 2025
Large Language Diffusion Models Shen Nie Fengqi Zhu Zebin You Xiaolu Zhang Jingyang Ou Jun Hu Jun Zhou Yankai Lin Ji-Rong Wen Chongxuan Li 110 15 0 14 Feb 2025
Scaling Laws for Predicting Downstream Performance in LLMs Yangyi Chen Binxuan Huang Yifan Gao Zhengyang Wang Jingfeng Yang Heng Ji LRM 43 8 0 11 Oct 2024
Improving Pretraining Data Using Perplexity Correlations Tristan Thrush Christopher Potts Tatsunori Hashimoto 32 17 0 09 Sep 2024
RegMix: Data Mixture as Regression for Language Model Pre-training Qian Liu Xiaosen Zheng Niklas Muennighoff Guangtao Zeng Longxu Dou Tianyu Pang Jing Jiang Min-Bin Lin MoE 71 40 1 01 Jul 2024
Linguistic Collapse: Neural Collapse in (Large) Language Models Robert Wu V. Papyan 48 12 0 28 May 2024
DeepSeek LLM: Scaling Open-Source Language Models with Longtermism DeepSeek-AI Xiao Bi : Xiao Bi Deli Chen Guanting Chen ... Yao Zhao Shangyan Zhou Shunfeng Zhou Qihao Zhu Yuheng Zou LRM ALM 139 306 0 05 Jan 2024
The Falcon Series of Open Language Models Ebtesam Almazrouei Hamza Alobeidli Abdulaziz Alshamsi Alessandro Cappelli Ruxandra-Aimée Cojocaru ... Quentin Malartic Daniele Mazzotta Badreddine Noune B. Pannier Guilherme Penedo AI4TS ALM 118 399 0 28 Nov 2023
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity Yao Lu Max Bartolo Alastair Moore Sebastian Riedel Pontus Stenetorp AILaw LRM 279 1,121 0 18 Apr 2021
Shortformer: Better Language Modeling using Shorter Inputs Ofir Press Noah A. Smith M. Lewis 225 89 0 31 Dec 2020