Scaling Laws for Neural Language Models

23 January 2020

Papers citing "Scaling Laws for Neural Language Models"

50 / 945 papers shown

Title
Hardware-Enabled Mechanisms for Verifying Responsible AI Development Aidan O'Gara Gabriel Kulp Will Hodgkins James Petrie Vincent Immler Aydin Aysu K. Basu S. Bhasin S. Picek Ankur Srivastava 19 0 0 02 Apr 2025
Advancing MoE Efficiency: A Collaboration-Constrained Routing (C2R) Strategy for Better Expert Parallelism Design Mohan Zhang Pingzhi Li Jie Peng Mufan Qiu Tianlong Chen MoE 45 0 0 02 Apr 2025
Geometric Median Matching for Robust k-Subset Selection from Noisy Data Anish Acharya Sujay Sanghavi Alexandros G. Dimakis Inderjit S Dhillon AAML 57 0 0 01 Apr 2025
Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Compute Jianhao Chen Zishuo Xun Bocheng Zhou Han Qi Qiaosheng Zhang ... Wei Hu Yuzhong Qu W. Ouyang Wanli Ouyang Shuyue Hu 74 0 0 01 Apr 2025
ToReMi: Topic-Aware Data Reweighting for Dynamic Pre-Training Data Selection Xiaoxuan Zhu Zhouhong Gu Baiqian Wu Suhang Zheng Tao Wang Tianyu Li Hongwei Feng Yanghua Xiao 42 0 0 01 Apr 2025
Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems? Kai Yan Yufei Xu Zhengyin Du Xuesong Yao Z. Wang Xiaowen Guo Jiecao Chen ReLM ELM LRM 95 3 0 01 Apr 2025
A Survey on Unlearnable Data Jiahao Li Yiqiang Chen Yunbing Xing Yang Gu Xiangyuan Lan AAML 58 0 0 30 Mar 2025
FastVAR: Linear Visual Autoregressive Modeling via Cached Token Pruning Hang Guo Yawei Li Taolin Zhang J. Wang Tao Dai Shu-Tao Xia Luca Benini 72 1 0 30 Mar 2025
Exploiting Mixture-of-Experts Redundancy Unlocks Multimodal Generative Abilities Raman Dutt Harleen Hanspal Guoxuan Xia Petru-Daniel Tudosiu Alexander Black Yongxin Yang Steven G. McDonagh Sarah Parisot MoE 40 0 0 28 Mar 2025
A Large-Scale Vision-Language Dataset Derived from Open Scientific Literature to Advance Biomedical Generalist AI Alejandro Lozano M. W. Sun James Burgess Jeffrey Nirschl Christopher Polzak ... Xiaohan Wang Alfred Seunghoon Song Chiang Chia-Chun Robert Tibshirani Serena Yeung-Levy LM&MA 96 1 0 26 Mar 2025
A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications Jian-Yu Guan J. Wu J. Li Chuanqi Cheng Wei Yu Wu LM&MA 71 0 0 21 Mar 2025
Striving for Simplicity: Simple Yet Effective Prior-Aware Pseudo-Labeling for Semi-Supervised Ultrasound Image Segmentation Yaxiong Chen Yujie Wang Zixuan Zheng Jingliang Hu Yilei Shi Shengwu Xiong Xiao Xiang Zhu Lichao Mou 54 0 0 18 Mar 2025
A Generalist Hanabi Agent Arjun Vaithilingam Sudhakar Hadi Nekoei Mathieu Reymond Miao Liu Janarthanan Rajendran Sarath Chandar 157 0 0 17 Mar 2025
SuperBPE: Space Travel for Language Models Alisa Liu J. Hayase Valentin Hofmann Sewoong Oh Noah A. Smith Yejin Choi 43 3 0 17 Mar 2025
Florenz: Scaling Laws for Systematic Generalization in Vision-Language Models Julian Spravil Sebastian Houben Sven Behnke VLM 70 0 0 12 Mar 2025
MaskAttn-UNet: A Mask Attention-Driven Framework for Universal Low-Resolution Image Segmentation Anzhe Cheng Chenzhong Yin Yu Chang Heng Ping Shixuan Li Shahin Nazarian Paul Bogdan SSeg 86 0 0 11 Mar 2025
Attention Hijackers: Detect and Disentangle Attention Hijacking in LVLMs for Hallucination Mitigation Beitao Chen Xinyu Lyu Lianli Gao Jingkuan Song H. Shen 63 1 0 11 Mar 2025
ProtTeX: Structure-In-Context Reasoning and Editing of Proteins with Large Language Models Zicheng Ma Chuanliu Fan Zhicong Wang Zhenyu Chen Xiaohan Lin Y. Li Shihao Feng Jun Zhang Ziqiang Cao Y. Gao 43 0 0 11 Mar 2025
MoFE: Mixture of Frozen Experts Architecture Jean Seo Jaeyoon Kim Hyopil Shin MoE 161 0 0 09 Mar 2025
WildIFEval: Instruction Following in the Wild Gili Lior Asaf Yehudai Ariel Gera L. Ein-Dor 68 0 0 09 Mar 2025
Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generation Mohit Pandey G. Subbaraj Artem Cherkasov Martin Ester Emmanuel Bengio AI4CE 81 1 0 08 Mar 2025
Multi-Task Reinforcement Learning Enables Parameter Scaling Reginald McLean Evangelos Chataroulas Jordan Terry Isaac Woungang Nariman Farsad P. S. Castro LRM 44 1 0 07 Mar 2025
Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs Ling Team B. Zeng C. Huang Chao Zhang Changxin Tian ... Zhaoxin Huan Zujie Wen Zhenhang Sun Zhuoxuan Du Z. He MoE ALM 109 2 0 07 Mar 2025
A Comprehensive LLM-powered Framework for Driving Intelligence Evaluation Shanhe You Xuewen Luo Xinhe Liang Jiashu Yu Chen Zheng Jiangtao Gong 69 0 0 07 Mar 2025
How can representation dimension dominate structurally pruned LLMs? Mingxue Xu Lisa Alazraki Danilo P. Mandic 56 0 0 06 Mar 2025
UIPE: Enhancing LLM Unlearning by Removing Knowledge Related to Forgetting Targets Wenyu Wang M. Zhang Xiaotian Ye Z. Z. Ren Z. Chen Pengjie Ren MU KELM 168 0 0 06 Mar 2025
AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletons Hongjie Fang Chenxi Wang Yiming Wang J. Chen Shangning Xia ... Xinyu Zhan Lixin Yang Weiming Wang Cewu Lu Hao-Shu Fang 82 1 0 05 Mar 2025
Towards Understanding Distilled Reasoning Models: A Representational Approach David D. Baek Max Tegmark LRM 80 3 0 05 Mar 2025
Generative Artificial Intelligence in Robotic Manipulation: A Survey Kun Zhang Peng Yun Jun Cen Junhao Cai DiDi Zhu ... Qifeng Chen Jia Pan Wei K. Zhang Bo Yang Hua Chen 59 1 0 05 Mar 2025
In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models David Ponce Thierry Etchegoyhen 72 1 0 03 Mar 2025
Neural ODE Transformers: Analyzing Internal Dynamics and Adaptive Fine-tuning Anh Tong Thanh Nguyen-Tang Dongeun Lee Duc Nguyen Toan M. Tran David Hall Cheongwoong Kang Jaesik Choi 35 0 0 03 Mar 2025
Predictive Data Selection: The Data That Predicts Is the Data That Teaches Kashun Shum Y. Huang Hongjian Zou Qi Ding Yixuan Liao X. Chen Qian Liu Junxian He 67 2 0 02 Mar 2025
Triple Phase Transitions: Understanding the Learning Dynamics of Large Language Models from a Neuroscience Perspective Yuko Nakagi Keigo Tada Sota Yoshino Shinji Nishimoto Yu Takagi LRM 37 0 0 28 Feb 2025
PaCA: Partial Connection Adaptation for Efficient Fine-Tuning Sunghyeon Woo Sol Namkung Sunwoo Lee Inho Jeong Beomseok Kim Dongsuk Jeon 39 0 0 28 Feb 2025
FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute Sotiris Anagnostidis Gregor Bachmann Yeongmin Kim Jonas Kohler Markos Georgopoulos A. Sanakoyeu Yuming Du Albert Pumarola Ali K. Thabet Edgar Schönfeld 92 0 0 27 Feb 2025
Shh, don't say that! Domain Certification in LLMs Cornelius Emde Alasdair Paren Preetham Arvind Maxime Kayser Tom Rainforth Thomas Lukasiewicz Bernard Ghanem Philip H. S. Torr Adel Bibi 50 1 0 26 Feb 2025
CAMEx: Curvature-aware Merging of Experts Dung V. Nguyen Minh H. Nguyen Luc Q. Nguyen R. Teo T. Nguyen Linh Duy Tran MoMe 98 2 0 26 Feb 2025
Nexus-O: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision Che Liu Yingji Zhang D. Zhang Weijie Zhang Chenggong Gong ... André Freitas Qifan Wang Z. Xu Rongjuncheng Zhang Yong Dai AuLLM 76 0 0 26 Feb 2025
Kanana: Compute-efficient Bilingual Language Models Kanana LLM Team Yunju Bak Hojin Lee Minho Ryu Jiyeon Ham ... Daniel Lee Minchul Lee M. Lee Shinbok Lee Gaeun Seo 90 1 0 26 Feb 2025
UQABench: Evaluating User Embedding for Prompting LLMs in Personalized Question Answering L. Liu Shilei Liu Yujin Yuan Y. Zhang Bencheng Yan ... Di Wang Wenbo Su Pengjie Wang Jian Xu Bo Zheng 51 1 0 26 Feb 2025
Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization Taishi Nakamura Takuya Akiba Kazuki Fujii Yusuke Oda Rio Yokota Jun Suzuki MoMe MoE 91 1 0 26 Feb 2025
From Small to Large Language Models: Revisiting the Federalist Papers So Won Jeong Veronika Rockova 37 0 0 25 Feb 2025
Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning Xinghao Chen Zhijing Sun Wenjin Guo Miaoran Zhang Yanjun Chen ... Hui Su Yijie Pan Dietrich Klakow Wenjie Li Xiaoyu Shen LRM 56 5 0 25 Feb 2025
Advantage-Guided Distillation for Preference Alignment in Small Language Models Shiping Gao Fanqi Wan Jiajian Guo Xiaojun Quan Qifan Wang ALM 58 0 0 25 Feb 2025
TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking Shahriar Kabir Nahin R. N. Nandi Sagor Sarker Quazi Sarwar Muhtaseem Md. Kowsher Apu Chandraw Shill Md Ibrahim Mehadi Hasan Menon Tareq Al Muntasir Firoj Alam 68 0 0 24 Feb 2025
LongAttn: Selecting Long-context Training Data via Token-level Attention Longyun Wu Dawei Zhu Guangxiang Zhao Zhuocheng Yu Junfeng Ran Xiangyu Wong Lin Sun Sujian Li 43 0 0 24 Feb 2025
Hyperspherical Normalization for Scalable Deep Reinforcement Learning Hojoon Lee Youngdo Lee Takuma Seno Donghu Kim Peter Stone Jaegul Choo 63 1 0 24 Feb 2025
Encryption-Friendly LLM Architecture Donghwan Rho Taeseong Kim Minje Park Jung Woo Kim Hyunsik Chae Jung Hee Cheon Ernest K. Ryu 54 2 0 24 Feb 2025
Contextualizing Search Queries In-Context Learning for Conversational Rewriting with LLMs Raymond Wilson Chase Carter Cole Graham 74 0 0 24 Feb 2025
COSMOS: A Hybrid Adaptive Optimizer for Memory-Efficient Training of LLMs Liming Liu Zhenghao Xu Zixuan Zhang Hao Kang Zichong Li Chen Liang Weizhu Chen T. Zhao 125 1 0 24 Feb 2025