Scaling Laws for Neural Language Models

23 January 2020

Papers citing "Scaling Laws for Neural Language Models"

50 / 949 papers shown

Title
Kimi k1.5: Scaling Reinforcement Learning with LLMs Kimi Team Angang Du Bofei Gao Bowei Xing Changjiu Jiang ... Zhilin Yang Zhiqi Huang Zihao Huang Ziyao Xu Z. Yang VLM ALM OffRL AI4TS LRM 106 141 0 22 Jan 2025
Physics of Skill Learning Ziming Liu Yizhou Liu Eric J. Michaud Jeff Gore Max Tegmark 46 1 0 21 Jan 2025
FOCUS: First Order Concentrated Updating Scheme Yizhou Liu Ziming Liu Jeff Gore ODL 108 1 0 21 Jan 2025
Human-like conceptual representations emerge from language prediction Ningyu Xu Qi Zhang Chao Du Qiang Luo Xipeng Qiu Xuanjing Huang Menghan Zhang 70 0 0 21 Jan 2025
Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling Zhenyu Hou Xin Lv Rui Lu J. Zhang Y. Li Zijun Yao Juanzi Li J. Tang Yuxiao Dong OffRL LRM ReLM 61 20 0 20 Jan 2025
QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks Yaojie Hu Qiang Zhou Qihong Chen Xiaopeng Li Linbo Liu Dejiao Zhang Amit Kachroo Talha Oz Omer Tripp 66 4 0 20 Jan 2025
Geometric Median (GM) Matching for Robust Data Pruning Anish Acharya Inderjit S Dhillon Sujay Sanghavi AAML 59 0 0 20 Jan 2025
LUT-DLA: Lookup Table as Efficient Extreme Low-Bit Deep Learning Accelerator Guoyu Li Shengyu Ye C. L. P. Chen Yang Wang Fan Yang Ting Cao Cheng Liu Mohamed M. Sabry Mao Yang MQ 137 0 0 18 Jan 2025
A Comprehensive Survey of Foundation Models in Medicine Wasif Khan Seowung Leem Kyle B. See Joshua K. Wong Shaoting Zhang R. Fang AI4CE LM&MA VLM 105 18 0 17 Jan 2025
CrisisSense-LLM: Instruction Fine-Tuned Large Language Model for Multi-label Social Media Text Classification in Disaster Informatics Kai Yin Chengkai Liu Ali Mostafavi Xia Hu 59 9 0 17 Jan 2025
O1 Replication Journey -- Part 3: Inference-time Scaling for Medical Reasoning Zhongzhen Huang Gui Geng Shengyi Hua Zhen Huang Haoyang Zou S. Zhang Pengfei Liu Xiaofan Zhang LRM 38 10 0 11 Jan 2025
Multi-Step Reasoning in Korean and the Emergent Mirage Guijin Son Hyunwoo Ko Dasol Choi LRM ReLM 65 0 0 10 Jan 2025
Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limit Oleg Filatov Jan Ebert Jiangtao Wang Stefan Kesselheim 36 3 0 10 Jan 2025
Audio-Language Datasets of Scenes and Events: A Survey Gijs Wijngaard Elia Formisano Michele Esposito M. Dumontier 81 2 0 10 Jan 2025
Generative AI for Cel-Animation: A Survey Yunlong Tang Junjia Guo Pinxin Liu Zhiyuan Wang Hang Hua ... Jing Bi Mingqian Feng X. Li Zeliang Zhang Chenliang Xu VGen 88 7 0 08 Jan 2025
Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation Yuhui Zhang Yuchang Su Yiming Liu Xiaohan Wang James Burgess ... Josiah Aklilu Alejandro Lozano Anjiang Wei Ludwig Schmidt Serena Yeung-Levy 61 3 0 06 Jan 2025
Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning Zhen Li Yupeng Su Runming Yang C. Xie Z. Wang Zhongwei Xie Ngai Wong Hongxia Yang MQ LRM 46 3 0 06 Jan 2025
Scaling Laws for Floating Point Quantization Training X. Sun Shuaipeng Li Ruobing Xie Weidong Han Kan Wu ... Yangyu Tao Zhanhui Kang C. Xu Di Wang Jie Jiang MQ AIFin 60 0 0 05 Jan 2025
ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use Junjie Ye Zhengyin Du Xuesong Yao Weijian Lin Yufei Xu ... Siyu Yuan Tao Gui Qi Zhang Xuanjing Huang Jiecao Chen 49 0 0 05 Jan 2025
Explicit vs. Implicit: Investigating Social Bias in Large Language Models through Self-Reflection Yachao Zhao Bo Wang Yan Wang 50 2 0 04 Jan 2025
Large Language Monkeys: Scaling Inference Compute with Repeated Sampling Bradley Brown Jordan Juravsky Ryan Ehrlich Ronald Clark Quoc V. Le Christopher Ré Azalia Mirhoseini ALM LRM 81 220 0 03 Jan 2025
The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better Scott Geng Cheng-Yu Hsieh Vivek Ramanujan Matthew Wallingford Chun-Liang Li Pang Wei Koh Ranjay Krishna DiffM 68 6 0 03 Jan 2025
Mathematical Language Models: A Survey W. Liu Hanglei Hu Jie Zhou Yuyang Ding Junsong Li ... Mengliang He Qin Chen Bo Jiang Aimin Zhou Liang He LRM 79 12 0 03 Jan 2025
Lost-in-Distance: Impact of Contextual Proximity on LLM Performance in Graph Tasks Hamed Firooz Maziar Sanjabi Wenlong Jiang Xiaoling Zhai 65 3 0 03 Jan 2025
Towards Precise Scaling Laws for Video Diffusion Transformers Yuanyang Yin Yaqi Zhao Mingwu Zheng Ke Lin Jiarong Ou ... Pengfei Wan Di Zhang Baoqun Yin Wentao Zhang Kun Gai 124 2 0 03 Jan 2025
KARPA: A Training-free Method of Adapting Knowledge Graph as References for Large Language Model's Reasoning Path Aggregation Siyuan Fang Kaijing Ma Tianyu Zheng Xinrun Du Ningxuan Lu Ge Zhang Qingkun Tang RALM KELM LRM 146 1 0 31 Dec 2024
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models Yulei Qin Yuncheng Yang Pengcheng Guo Gang Li Hang Shao Yuchen Shi Zihan Xu Yun Gu Ke Li Xing Sun ALM 90 12 0 31 Dec 2024
Facilitating large language model Russian adaptation with Learned Embedding Propagation Mikhail Tikhomirov D. Chernyshev 38 1 0 31 Dec 2024
Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism Tim Tsz-Kit Lau Weijian Li Chenwei Xu Han Liu Mladen Kolar 147 0 0 30 Dec 2024
Grams: Gradient Descent with Adaptive Momentum Scaling Yang Cao Xiaoyu Li Zhao-quan Song ODL 87 2 0 22 Dec 2024
MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis Ho Kei Cheng Masato Ishii Akio Hayakawa Takashi Shibuya A. Schwing Yuki Mitsufuji VGen 126 12 0 19 Dec 2024
Next Patch Prediction for Autoregressive Visual Generation Yatian Pang Peng Jin Shuo Yang Bin Lin Bin Zhu ... Liuhan Chen Francis E. H. Tay Ser-Nam Lim Harry Yang Li Yuan 120 8 0 19 Dec 2024
Wearable Accelerometer Foundation Models for Health via Knowledge Distillation Salar Abbaspourazad Anshuman Mishra Joseph D. Futoma Andrew C. Miller Ian Shapiro 88 0 0 15 Dec 2024
A recent evaluation on the performance of LLMs on radiation oncology physics using questions of randomly shuffled options Peilong Wang J. Holmes Z. Liu Dequan Chen Tianming Liu Jiajian Shen W. Liu LRM ELM LM&MA 82 0 0 14 Dec 2024
Vision Transformers for Efficient Indoor Pathloss Radio Map Prediction Rafayel Mkrtchyan Edvard Ghukasyan Khoren Petrosyan Hrant Khachatrian Theofanis P. Raptis 81 0 0 12 Dec 2024
Sloth: scaling laws for LLM skills to predict multi-benchmark performance across families Felipe Maia Polo S. Kamath S Leshem Choshen Yuekai Sun Mikhail Yurochkin 89 6 0 09 Dec 2024
Context Clues: Evaluating Long Context Models for Clinical Prediction Tasks on EHRs Michael Wornow Suhana Bedi Miguel Angel Fuentes Hernandez E. Steinberg Jason Alan Fries Christopher Ré Sanmi Koyejo N. Shah 95 4 0 09 Dec 2024
Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization Dongwei Wang Huanrui Yang MQ 85 1 0 08 Dec 2024
Turing Representational Similarity Analysis (RSA): A Flexible Method for Measuring Alignment Between Human and Artificial Intelligence Mattson Ogg Ritwik Bose Jamie Scharf Christopher Ratto Michael Wolmetz 83 2 0 30 Nov 2024
Puzzle: Distillation-Based NAS for Inference-Optimized LLMs Akhiad Bercovich Tomer Ronen Talor Abramovich Nir Ailon Nave Assaf ... Ido Shahaf Oren Tropp Omer Ullman Argov Ran Zilberstein Ran El-Yaniv 77 1 0 28 Nov 2024
COAP: Memory-Efficient Training with Correlation-Aware Gradient Projection Jinqi Xiao S. Sang Tiancheng Zhi Jing Liu Qing Yan Linjie Luo Bo Yuan Bo Yuan VLM 86 1 0 26 Nov 2024
Cautious Optimizers: Improving Training with One Line of Code Kaizhao Liang Lizhang Chen B. Liu Qiang Liu ODL 108 5 0 25 Nov 2024
SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE Yongwei Chen Yushi Lan Shangchen Zhou Tengfei Wang Xingang Pan 100 5 0 25 Nov 2024
Tra-MoE: Learning Trajectory Prediction Model from Multiple Domains for Adaptive Policy Conditioning Jiange Yang Haoyi Zhu Y. Wang Gangshan Wu Tong He Limin Wang 100 2 0 21 Nov 2024
Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training Jared Fernandez Luca Wehrstedt Leonid Shamis Mostafa Elhoushi Kalyan Saladi Yonatan Bisk Emma Strubell Jacob Kahn 198 3 0 20 Nov 2024
Training Bilingual LMs with Data Constraints in the Targeted Language Skyler Seto Maartje ter Hoeve He Bai Natalie Schluter David Grangier 81 0 0 20 Nov 2024
MAS-Attention: Memory-Aware Stream Processing for Attention Acceleration on Resource-Constrained Edge Devices Mohammadali Shakerdargah Shan Lu Chao Gao Di Niu 70 0 0 20 Nov 2024
VILA-M3: Enhancing Vision-Language Models with Medical Expert Knowledge Vishwesh Nath Wenqi Li Dong Yang Andriy Myronenko Mingxin Zheng ... Holger Roth Daguang Xu Baris Turkbey Holger Roth Daguang Xu VLM 97 4 0 19 Nov 2024
Unsupervised detection of semantic correlations in big data Santiago Acevedo Alex Rodriguez A. Laio 70 2 0 04 Nov 2024
Context Parallelism for Scalable Million-Token Inference Amy Yang Jingyi Yang Aya Ibrahim Xinfeng Xie Bangsheng Tang Grigory Sizov Jeremy Reizenstein Jongsoo Park Jianyu Huang MoE LRM 62 5 0 04 Nov 2024