Measuring Massive Multitask Language Understanding

7 September 2020

Papers citing "Measuring Massive Multitask Language Understanding"

50 / 956 papers shown

Title
RegMix: Data Mixture as Regression for Language Model Pre-training Qian Liu Xiaosen Zheng Niklas Muennighoff Guangtao Zeng Longxu Dou Tianyu Pang Jing Jiang Min Lin MoE 74 41 1 01 Jul 2024
Iterative Nash Policy Optimization: Aligning LLMs with General Preferences via No-Regret Learning Yuheng Zhang Dian Yu Baolin Peng Linfeng Song Ye Tian Mingyue Huo Nan Jiang Haitao Mi Dong Yu 37 15 0 30 Jun 2024
It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization Bingdong Li Zixiang Di Yanting Yang Hong Qian Peng Yang Hao Hao Ke Tang Aimin Zhou MoMe 32 5 0 29 Jun 2024
CMMaTH: A Chinese Multi-modal Math Skill Evaluation Benchmark for Foundation Models Zhong-Zhi Li Ming-Liang Zhang Fei Yin Zhi-Long Ji Jin-Feng Bai Zhen-Ru Pan Fan-Hu Zeng Jian Xu Jia-Xin Zhang Cheng-Lin Liu ELM 51 12 0 28 Jun 2024
Evaluating Copyright Takedown Methods for Language Models Boyi Wei Weijia Shi Yangsibo Huang Noah A. Smith Chiyuan Zhang Luke Zettlemoyer Kai Li Peter Henderson 54 20 0 26 Jun 2024
PaCoST: Paired Confidence Significance Testing for Benchmark Contamination Detection in Large Language Models Huixuan Zhang Yun Lin Xiaojun Wan 53 0 0 26 Jun 2024
RouteLLM: Learning to Route LLMs with Preference Data Isaac Ong Amjad Almahairi Vincent Wu Wei-Lin Chiang Tianhao Wu Joseph E. Gonzalez M. W. Kadous Ion Stoica 81 77 0 26 Jun 2024
VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbation Kun Qian Shunji Wan Claudia Tang Youzhi Wang Xuanming Zhang Maximillian Chen Zhou Yu AAML 52 8 0 25 Jun 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 62 14 0 24 Jun 2024
AnnotatedTables: A Large Tabular Dataset with Language Model Annotations Yaojie Hu Ilias Fountalis Jin Tian N. Vasiloglou LMTD 41 4 0 24 Jun 2024
Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation Yuchen Yang Yingdong Shi Cheems Wang Xiantong Zhen Yuxuan Shi Jun Xu 42 1 0 24 Jun 2024
Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging Deyuan Liu Zengchang Qin Han Wang Zhao Yang Zecheng Wang ... Zhao Lv Zhiying Tu Dianhui Chu Bo Li Dianbo Sui 36 2 0 24 Jun 2024
Towards Scalable Exact Machine Unlearning Using Parameter-Efficient Fine-Tuning Somnath Basu Roy Chowdhury Krzysztof Choromanski Arijit Sehanobish Avinava Dubey Snigdha Chaturvedi MU 61 7 0 24 Jun 2024
SEAM: A Stochastic Benchmark for Multi-Document Tasks Gili Lior Avi Caciularu Arie Cattan Shahar Levy Ori Shapira Gabriel Stanovsky RALM 42 4 0 23 Jun 2024
AudioBench: A Universal Benchmark for Audio Large Language Models Bin Wang Xunlong Zou Geyu Lin Shri Kiran Srinivasan Zhuohan Liu Wenyu Zhang Zhengyuan Liu AiTi Aw Nancy F. Chen AuLLM ELM LM&MA 92 23 0 23 Jun 2024
Crosslingual Capabilities and Knowledge Barriers in Multilingual Large Language Models Lynn Chua Badih Ghazi Yangsibo Huang Pritish Kamath Ravi Kumar Pasin Manurangsi Amer Sinha Chulin Xie Chiyuan Zhang 69 1 0 23 Jun 2024
Chain-of-Probe: Examining the Necessity and Accuracy of CoT Step-by-Step Zezhong Wang Xingshan Zeng Weiwen Liu Yufei Wang Liangyou Li Yasheng Wang Lifeng Shang Xin Jiang Qun Liu Kam-Fai Wong LRM 70 4 0 23 Jun 2024
The Music Maestro or The Musically Challenged, A Massive Music Evaluation Benchmark for Large Language Models Jiajia Li Lu Yang Mingni Tang Cong Chen Zuchao Li Ping Wang Hai Zhao LM&MA 46 4 0 22 Jun 2024
RuleR: Improving LLM Controllability by Rule-based Data Recycling Ming Li Han Chen Chenguang Wang Dang Nguyen Dianqi Li Dinesh Manocha 33 7 0 22 Jun 2024
Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph Roman Vashurin Ekaterina Fadeeva Artem Vazhentsev Akim Tsvigun Daniil Vasilev ... Timothy Baldwin Timothy Baldwin Maxim Panov Artem Shelmanov Artem Shelmanov HILM 68 10 0 21 Jun 2024
Do Large Language Models Exhibit Cognitive Dissonance? Studying the Difference Between Revealed Beliefs and Stated Answers Manuel Mondal Ljiljana Dolamic Gérôme Bovet Philippe Cudré-Mauroux Julien Audiffren 46 2 0 21 Jun 2024
Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing Han Jiang Xiaoyuan Yi Zhihua Wei Shu Wang Xing Xie Xing Xie ALM ELM 59 5 0 20 Jun 2024
SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal Tinghao Xie Xiangyu Qi Yi Zeng Yangsibo Huang Udari Madhushani Sehwag ... Bo Li Kai Li Danqi Chen Peter Henderson Prateek Mittal ALM ELM 58 57 0 20 Jun 2024
Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometrics Seungbeen Lee Seungwon Lim Seungju Han Giyeong Oh Hyungjoo Chae ... Beong-woo Kwak Yeonsoo Lee Dongha Lee Jinyoung Yeo Youngjae Yu 44 8 0 20 Jun 2024
Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges Aman Singh Thakur Kartik Choudhary Venkat Srinik Ramayapally Sankaran Vaidyanathan Dieuwke Hupkes ELM ALM 63 57 0 18 Jun 2024
OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI Zhen Huang Zengzhi Wang Shijie Xia Xuefeng Li Haoyang Zou ... Yuxiang Zheng Shaoting Zhang Dahua Lin Yu Qiao Pengfei Liu ELM LRM 51 30 0 18 Jun 2024
Split, Unlearn, Merge: Leveraging Data Attributes for More Effective Unlearning in LLMs S. Kadhe Farhan Ahmed Dennis Wei Nathalie Baracaldo Inkit Padhi MoMe MU 33 7 0 17 Jun 2024
Input Conditioned Graph Generation for Language Agents Lukas Vierling Jie Fu Kai Chen LLMAG 63 2 0 17 Jun 2024
Are Large Language Models True Healthcare Jacks-of-All-Trades? Benchmarking Across Health Professions Beyond Physician Exams Zheheng Luo Chenhan Yuan Qianqian Xie Sophia Ananiadou ELM AI4MH LM&MA 51 0 0 17 Jun 2024
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion Chenghao Fan Zhenyi Lu Wei Wei Jie Tian Xiaoye Qu Dangyang Chen Yu Cheng MoMe 55 5 0 17 Jun 2024
Exploring Safety-Utility Trade-Offs in Personalized Language Models Anvesh Rao Vijjini Somnath Basu Roy Chowdhury Snigdha Chaturvedi 59 7 0 17 Jun 2024
ShareLoRA: Parameter Efficient and Robust Large Language Model Fine-tuning via Shared Low-Rank Adaptation Yurun Song Junchen Zhao Ian G. Harris Sangeetha Abdu Jyothi 32 3 0 16 Jun 2024
Applications of Generative AI in Healthcare: algorithmic, ethical, legal and societal considerations Onyekachukwu R. Okonji Kamol Yunusov Bonnie Gordon MedIm 46 3 0 15 Jun 2024
SciEx: Benchmarking Large Language Models on Scientific Exams with Human Expert Grading and Automatic Grading Tu Anh Dinh Carlos Mullov Leonard Barmann Zhaolin Li Danni Liu ... Michael Beigl Rainer Stiefelhagen Carsten Dachsbacher Klemens Bohm Jan Niehues ELM 45 8 0 14 Jun 2024
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack Yuri Kuratov Aydar Bulatov Petr Anokhin Ivan Rodkin Dmitry Sorokin Artyom Sorokin Andrey Kravchenko RALM ALM LRM ReLM ELM 53 61 0 14 Jun 2024
What is the best model? Application-driven Evaluation for Large Language Models Shiguo Lian Kaikai Zhao Xinhui Liu Xuejiao Lei Bikun Yang Wenjing Zhang Kai Wang Zhaoxiang Liu ALM ELM 43 2 0 14 Jun 2024
A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations Jinqiang Wang Huansheng Ning Yi Peng Qikai Wei Daniel Tesfai Wenwei Mao Tao Zhu Runhe Huang LM&MA AI4MH ELM 54 5 0 14 Jun 2024
Benchmarking Generative Models on Computational Thinking Tests in Elementary Visual Programming Victor-Alexandru Pădurean Adish Singla ELM 59 3 0 14 Jun 2024
SciKnowEval: Evaluating Multi-level Scientific Knowledge of Large Language Models Kehua Feng Keyan Ding Weijie Wang Xiang Zhuang Zeyuan Wang Ming Qin Yu Zhao Jianhua Yao Qiang Zhang H. Chen ELM 50 6 0 13 Jun 2024
Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing Zhangchen Xu Fengqing Jiang Luyao Niu Yuntian Deng Radha Poovendran Yejin Choi Bill Yuchen Lin SyDa 59 129 0 12 Jun 2024
An Empirical Study of Mamba-based Language Models R. Waleffe Wonmin Byeon Duncan Riach Brandon Norick V. Korthikanti ... Vartika Singh Jared Casper Jan Kautz Mohammad Shoeybi Bryan Catanzaro 63 66 0 12 Jun 2024
Are Large Language Models Good Statisticians? Yizhang Zhu Shiyin Du Boyan Li Yuyu Luo Nan Tang ELM 48 15 0 12 Jun 2024
CS-Bench: A Comprehensive Benchmark for Large Language Models towards Computer Science Mastery Xiaoshuai Song Muxi Diao Guanting Dong Zhengyang Wang Yujia Fu ... Yejie Wang Zhuoma Gongque Jianing Yu Qiuna Tan Weiran Xu ELM 60 11 0 12 Jun 2024
OLMES: A Standard for Language Model Evaluations Yuling Gu Oyvind Tafjord Bailey Kuehl Dany Haddad Jesse Dodge Hannaneh Hajishirzi ELM 45 14 0 12 Jun 2024
Crayon: Customized On-Device LLM via Instant Adapter Blending and Edge-Server Hybrid Inference Jihwan Bang Juntae Lee Kyuhong Shim Seunghan Yang Simyung Chang 39 5 0 11 Jun 2024
AI Sandbagging: Language Models can Strategically Underperform on Evaluations Teun van der Weij Felix Hofstätter Ollie Jaffe Samuel F. Brown Francis Rhys Ward ELM 52 22 0 11 Jun 2024
Scaling Large Language Model-based Multi-Agent Collaboration Chen Qian Zihao Xie YiFei Wang Wei Liu Yufan Dang ... Zhuoyun Du Weize Chen Cheng Yang Zhiyuan Liu Maosong Sun AI4CE LLMAG LM&Ro 69 47 0 11 Jun 2024
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling Liliang Ren Yang Liu Yadong Lu Yelong Shen Chen Liang Weizhu Chen Mamba 77 57 0 11 Jun 2024
Can I understand what I create? Self-Knowledge Evaluation of Large Language Models Zhiquan Tan Lai Wei Jindong Wang Xing Xie Weiran Huang ELM LRM 35 5 0 10 Jun 2024
CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark David Romero Chenyang Lyu Haryo Akbarianto Wibowo Teresa Lynn Injy Hamed ... Oana Ignat Joan Nwatu Rada Mihalcea Thamar Solorio Alham Fikri Aji 48 26 0 10 Jun 2024