FOFO: A Benchmark to Evaluate LLMs' Format-Following Capability

FOFO: A Benchmark to Evaluate LLMs' Format-Following Capability

28 February 2024

Ran Xu

Papers citing "FOFO: A Benchmark to Evaluate LLMs' Format-Following Capability"

13 / 13 papers shown

Title
Learning to Generate Structured Output with Schema Reinforcement Learning Yaojie Lu Haolun Li Xin Cong Zhong Zhang Yesai Wu Yankai Lin Zhiyuan Liu Fangming Liu Maosong Sun 44 1 0 26 Feb 2025
SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Models Jiale Cheng Xiao-Chang Liu C. Wang Xiaotao Gu Yaojie Lu Dan Zhang Yuxiao Dong J. Tang Hongning Wang Minlie Huang LRM 126 3 0 16 Dec 2024
Do LLMs "know" internally when they follow instructions? Juyeon Heo Christina Heinze-Deml Oussama Elachqar Shirley Ren Udhay Nallasamy Andy Miller Kwan Ho Ryan Chan Jaya Narain 51 5 0 18 Oct 2024
Do LLMs estimate uncertainty well in instruction-following? Juyeon Heo Miao Xiong Christina Heinze-Deml Jaya Narain ELM 52 3 0 18 Oct 2024
Evaluating the Instruction-following Abilities of Language Models using Knowledge Tasks Rudra Murthy Prince Kumar Praveen Venkateswaran Danish Contractor KELM ALM ELM 34 1 0 16 Oct 2024
LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs Do Xuan Long Hai Nguyen Ngoc Tiviatis Sim Hieu Dao Shafiq R. Joty Kenji Kawaguchi Nancy F. Chen Min-Yen Kan 34 7 0 16 Aug 2024
CFBench: A Comprehensive Constraints-Following Benchmark for LLMs Leo Micklem Yan-Bin Shen Wenjing Luo Yan Zhang Hao Liang ... Weipeng Chen Bin Cui Blair Thornton Wentao Zhang Zenan Zhou ELM 81 16 0 02 Aug 2024
A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations Md Tahmid Rahman Laskar Sawsan Alqahtani M Saiful Bari Mizanur Rahman Mohammad Abdullah Matin Khan ... Chee Wei Tan Md. Rizwan Parvez Enamul Hoque Shafiq R. Joty Jimmy Huang ELM ALM 29 28 0 04 Jul 2024
Evaluation of Instruction-Following Ability for Large Language Models on Story-Ending Generation Rem Hida Junki Ohmura Toshiyuki Sekiya ELM 32 0 0 24 Jun 2024
RuleR: Improving LLM Controllability by Rule-based Data Recycling Ming Li Han Chen Chenguang Wang Dang Nguyen Dianqi Li Dinesh Manocha 28 6 0 22 Jun 2024
Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models Hengyi Wang Haizhou Shi Shiwei Tan Weiyi Qin Wenyuan Wang Tunyu Zhang A. Nambi T. Ganu Hao Wang 71 14 0 17 Jun 2024
Evalverse: Unified and Accessible Library for Large Language Model Evaluation Jihoo Kim Wonho Song Dahyun Kim Yunsu Kim Yungi Kim Chanjun Park ELM 63 3 0 01 Apr 2024
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 319 11,953 0 04 Mar 2022