FSPO: Few-Shot Preference Optimization of Synthetic Preference Data in LLMs Elicits Effective Personalization to Real Users

FSPO: Few-Shot Preference Optimization of Synthetic Preference Data in LLMs Elicits Effective Personalization to Real Users

26 February 2025

Tatsunori Hashimoto

Papers citing "FSPO: Few-Shot Preference Optimization of Synthetic Preference Data in LLMs Elicits Effective Personalization to Real Users"

17 / 17 papers shown

Title
Amulet: Putting Complex Multi-Turn Conversations on the Stand with LLM Juries Sahana Ramnath Anurag Mudgil Brihi Joshi Skyler Hallinan Xiang Ren 22 0 0 26 May 2025
LoRe: Personalizing LLMs via Low-Rank Reward Modeling Avinandan Bose Zhihan Xiong Yuejie Chi Simon S. Du Lin Xiao Maryam Fazel 51 0 0 20 Apr 2025
Benchmarking Distributional Alignment of Large Language Models Nicole Meister Carlos Guestrin Tatsunori Hashimoto ALM 39 4 0 08 Nov 2024
Context Parallelism for Scalable Million-Token Inference Amy Yang Jingyi Yang Aya Ibrahim Xinfeng Xie Bangsheng Tang Grigory Sizov Jeremy Reizenstein Jongsoo Park Jianyu Huang MoE LRM 83 6 0 04 Nov 2024
Distributional Preference Alignment of LLMs via Optimal Transport Igor Melnyk Youssef Mroueh Brian M. Belgodere Mattia Rigotti Apoorva Nitsure Mikhail Yurochkin Kristjan Greenewald Jirí Navrátil Jerret Ross 69 12 0 09 Jun 2024
Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators Yann Dubois Balázs Galambosi Percy Liang Tatsunori Hashimoto ALM 64 359 0 06 Apr 2024
Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models Haoran Li Qingxiu Dong Zhengyang Tang Chaojun Wang Xingxing Zhang ... Wei Lu Zhifang Sui Benyou Wang Wai Lam Furu Wei SyDa 65 58 0 20 Feb 2024
Self-Rewarding Language Models Weizhe Yuan Richard Yuanzhe Pang Kyunghyun Cho Xian Li Sainbayar Sukhbaatar Jing Xu Jason Weston ReLM SyDa ALM LRM 275 312 0 18 Jan 2024
Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF Anand Siththaranjan Cassidy Laidlaw Dylan Hadfield-Menell 68 62 0 13 Dec 2023
A General Theoretical Paradigm to Understand Learning from Human Preferences M. G. Azar Mark Rowland Bilal Piot Daniel Guo Daniele Calandriello Michal Valko Rémi Munos 99 580 0 18 Oct 2023
Reinforced Self-Training (ReST) for Language Modeling Çağlar Gülçehre T. Paine S. Srinivasan Ksenia Konyushkova L. Weerts ... Chenjie Gu Wolfgang Macherey Arnaud Doucet Orhan Firat Nando de Freitas OffRL 85 291 0 17 Aug 2023
AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback Yann Dubois Xuechen Li Rohan Taori Tianyi Zhang Ishaan Gulrajani Jimmy Ba Carlos Guestrin Percy Liang Tatsunori B. Hashimoto ALM 83 569 0 22 May 2023
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 445 41,106 0 28 May 2020
Meta-Learning without Memorization Mingzhang Yin George Tucker Mingyuan Zhou Sergey Levine Chelsea Finn VLM 34 186 0 09 Dec 2019
Domain Randomization and Generative Models for Robotic Grasping Joshua Tobin Lukas Biewald Rocky Duan Marcin Andrychowicz Ankur Handa ... Bob McGrew Jonas Schneider Peter Welinder Wojciech Zaremba Pieter Abbeel OOD 61 175 0 17 Oct 2017
Proximal Policy Optimization Algorithms John Schulman Filip Wolski Prafulla Dhariwal Alec Radford Oleg Klimov OffRL 201 18,685 0 20 Jul 2017
Learning to reinforcement learn Jane X. Wang Z. Kurth-Nelson Dhruva Tirumala Hubert Soyer Joel Z Leibo Rémi Munos Charles Blundell D. Kumaran M. Botvinick OffRL 62 974 0 17 Nov 2016

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from. See our policy.