ScalingFilter: Assessing Data Quality through Inverse Utilization of Scaling Laws

15 August 2024

Papers citing "ScalingFilter: Assessing Data Quality through Inverse Utilization of Scaling Laws"

4 / 4 papers shown

Title
AttentionInfluence: Adopting Attention Head Influence for Weak-to-Strong Pretraining Data Selection Kai Hua Steven Wu Ge Zhang Ke Shen LRM 28 0 0 12 May 2025
The Vendi Score: A Diversity Evaluation Metric for Machine Learning Dan Friedman Adji Bousso Dieng EGVM 94 109 0 05 Oct 2022
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 246 4,489 0 23 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,821 0 17 Sep 2019