MarginSel : Max-Margin Demonstration Selection for LLMs

Large Language Models (LLMs) excel at few-shot learning via in-context learning (ICL). However, the effectiveness of ICL is often sensitive to the selection and ordering of demonstration examples. To address this, we present MarginSel: Max-Margin Demonstration Selection for LLMs, a two-step method that selects hard demonstration examples for the ICL prompt, adapting to each test instance. Our approach achieves 2-7% absolute improvement in F1-score across classification tasks, compared to a random selection of examples. We also provide theoretical insights and empirical evidence showing that MarginSel induces max-margin behavior in LLMs by effectively increasing the margin for hard examples, analogous to support vectors, thereby shifting the decision boundary in a beneficial direction.

@article{ambati2025_2506.06699,
  title={ MarginSel : Max-Margin Demonstration Selection for LLMs },
  author={ Rajeev Bhatt Ambati and James Lester and Shashank Srivastava and Snigdha Chaturvedi },
  journal={arXiv preprint arXiv:2506.06699},
  year={ 2025 }
}