Entropy Law: The Story Behind Data Compression and LLM Performance

Entropy Law: The Story Behind Data Compression and LLM Performance

9 July 2024

Hao Wang

Wei Guo

Papers citing "Entropy Law: The Story Behind Data Compression and LLM Performance"

7 / 7 papers shown

Title
ALinFiK: Learning to Approximate Linearized Future Influence Kernel for Scalable Third-Party LLM Data Valuation Yanzhou Pan Huawei Lin Yide Ran Jiamin Chen Xiaodong Yu Weijie Zhao Denghui Zhang Zhaozhuo Xu 56 1 0 02 Mar 2025
Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction Xiang Xu Hao Wang Wei Guo Lefei Zhang Wanshan Yang R. Yu Yixiao Liu Defu Lian Enhong Chen 99 7 0 22 Nov 2024
gzip Predicts Data-dependent Scaling Laws Rohan Pandey 57 11 0 26 May 2024
What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning Wei Liu Weihao Zeng Keqing He Yong Jiang Junxian He ALM 56 231 0 25 Dec 2023
MoDS: Model-oriented Data Selection for Instruction Tuning Qianlong Du Chengqing Zong Jiajun Zhang ALM 47 83 0 27 Nov 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 215 4,085 0 09 Jun 2023
GPT-4 Technical Report OpenAI OpenAI OpenAI Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad ... Shengjia Zhao Tianhao Zheng Juntang Zhuang William Zhuk Barret Zoph LLMAG MLLM 403 13,788 0 15 Mar 2023