Exploring ChatGPT's Ability to Rank Content: A Preliminary Study on Consistency with Human Preferences

14 March 2023

Xiangang Li

Papers citing "Exploring ChatGPT's Ability to Rank Content: A Preliminary Study on Consistency with Human Preferences"

30 / 30 papers shown

Title
Implementing Rational Choice Functions with LLMs and Measuring their Alignment with User Preferences Anna Karnysheva Christian Drescher Dietrich Klakow 29 0 0 22 Apr 2025
Context-Aware Deep Learning for Multi Modal Depression Detection Genevieve Lam Huang Dongyan Weisi Lin 35 80 0 26 Dec 2024
Explingo: Explaining AI Predictions using Large Language Models Alexandra Zytek Sara Pido Sarah Alnegheimish Laure Berti-Equille K. Veeramachaneni 74 1 0 06 Dec 2024
Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructions Bhuvanashree Murugadoss Christian Poelitz Ian Drosos Vu Le Nick McKenna Carina Negreanu Chris Parnin Advait Sarkar ELM ALM 35 13 0 16 Aug 2024
Themis: Towards Flexible and Interpretable NLG Evaluation Xinyu Hu Li Lin Mingqi Gao Xunjian Yin Xiaojun Wan ELM 34 6 0 26 Jun 2024
MD tree: a model-diagnostic tree grown on loss landscape Yefan Zhou Jianlong Chen Qinxue Cao Konstantin Schürholt Yaoqing Yang 33 2 0 24 Jun 2024
Language Models can Evaluate Themselves via Probability Discrepancy Tingyu Xia Bowen Yu Yuan Wu Yi-Ju Chang Chang Zhou ELM 37 4 0 17 May 2024
LLM-based NLG Evaluation: Current Status and Challenges Mingqi Gao Xinyu Hu Jie Ruan Xiao Pu Xiaojun Wan ELM LM&MA 65 29 0 02 Feb 2024
Leveraging Large Language Models for NLG Evaluation: Advances and Challenges Zhen Li Xiaohan Xu Tao Shen Can Xu Jia-Chen Gu Yuxuan Lai Chongyang Tao Shuai Ma LM&MA ELM 39 9 0 13 Jan 2024
The Eval4NLP 2023 Shared Task on Prompting Large Language Models as Explainable Metrics Christoph Leiter Juri Opitz Daniel Deutsch Yang Gao Rotem Dror Steffen Eger ALM LRM ELM 40 31 0 30 Oct 2023
CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation Weixiang Yan Yuchen Tian Yunzhe Li Qian Chen Wen Wang 34 35 0 08 Oct 2023
ChatGPT Hallucinates when Attributing Answers Guido Zuccon Bevan Koopman Razia Shaik RALM LRM HILM 55 26 0 17 Sep 2023
BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models Wei Qi Leong Jian Gang Ngui Yosephine Susanto Hamsawardhini Rengarajan Kengatharaiyer Sarveswaran William-Chandra Tjhi 26 9 0 12 Sep 2023
Generating Natural Language Queries for More Effective Systematic Review Screening Prioritisation Shuai Wang Harrisen Scells Martin Potthast Bevan Koopman Guido Zuccon 27 10 0 11 Sep 2023
A Preliminary Study of the Intrinsic Relationship between Complexity and Alignment Ying Zhao Yu Bowen Binyuan Hui Haiyang Yu Fei Huang Yongbin Li N. Zhang 44 23 0 10 Aug 2023
Wider and Deeper LLM Networks are Fairer LLM Evaluators Xinghua Zhang Yu Bowen Haiyang Yu Yangyu Lv Tingwen Liu Fei Huang Hongbo Xu Yongbin Li ALM 56 82 0 03 Aug 2023
Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs Zhikai Chen Haitao Mao Hang Li Wei Jin Haifang Wen ... Shuaiqiang Wang Dawei Yin Wenqi Fan Hui Liu Jiliang Tang AI4CE 57 266 0 07 Jul 2023
Towards Explainable Evaluation Metrics for Machine Translation Christoph Leiter Piyawat Lertvittayakumjorn M. Fomicheva Wei-Ye Zhao Yang Gao Steffen Eger ELM 30 13 0 22 Jun 2023
ChatGPT is a Remarkable Tool -- For Experts A. Azaria Rina Azoulay-Schwartz S. Reches 24 58 0 02 Jun 2023
A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation Xiaowei Huang Wenjie Ruan Wei Huang Gao Jin Yizhen Dong ... Sihao Wu Peipei Xu Dengyu Wu André Freitas Mustafa A. Mustafa ALM 45 82 0 19 May 2023
"HOT" ChatGPT: The promise of ChatGPT in detecting and discriminating hateful, offensive, and toxic comments on social media Lingyao Li Lizhou Fan Shubham Atreja Libby Hemphill AI4MH 47 84 0 20 Apr 2023
Human-like Summarization Evaluation with ChatGPT Mingqi Gao Jie Ruan Renliang Sun Xunjian Yin Shiping Yang Xiaojun Wan ALM AI4MH 23 125 0 05 Apr 2023
Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study Yi Chen Rui Wang Haiyun Jiang Shuming Shi Ruifeng Xu LM&MA 35 74 0 03 Apr 2023
Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Cases Yunjie Ji Yong Deng Yan Gong Yiping Peng Qiang Niu Lefei Zhang Baochang Ma Xiangang Li ALM 21 93 0 26 Mar 2023
Zero-Shot Cross-Lingual Summarization via Large Language Models Jiaan Wang Yunlong Liang Fandong Meng Beiqi Zou Zhixu Li Jianfeng Qu Jie Zhou ELM 29 28 0 28 Feb 2023
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 227 506 0 28 Sep 2022
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli Liane Lovitt John Kernion Amanda Askell Yuntao Bai ... Nicholas Joseph Sam McCandlish C. Olah Jared Kaplan Jack Clark 231 446 0 23 Aug 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 339 12,003 0 04 Mar 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 213 1,661 0 15 Oct 2021
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 298 1,610 0 18 Sep 2019