LLM Agent for Hyper-Parameter Optimization

Hyper-parameters are essential and critical for the performance of communication algorithms. However, current hyper-parameters tuning methods for warm-start particles swarm optimization with cross and mutation (WS-PSO-CM) algortihm for radio map-enabled unmanned aerial vehicle (UAV) trajectory and communication are primarily heuristic-based, exhibiting low levels of automation and unsatisfactory performance. In this paper, we design an large language model (LLM) agent for automatic hyper-parameters-tuning, where an iterative framework and model context protocol (MCP) are applied. In particular, the LLM agent is first setup via a profile, which specifies the mission, background, and output format. Then, the LLM agent is driven by the prompt requirement, and iteratively invokes WS-PSO-CM algorithm for exploration. Finally, the LLM agent autonomously terminates the loop and returns a set of hyper-parameters. Our experiment results show that the minimal sum-rate achieved by hyper-parameters generated via our LLM agent is significantly higher than those by both human heuristics and random generation methods. This indicates that an LLM agent with PSO knowledge and WS-PSO-CM algorithm background is useful in finding high-performance hyper-parameters.

@article{wang2025_2506.15167,
  title={ LLM Agent for Hyper-Parameter Optimization },
  author={ Wanzhe Wang and Jianqiu Peng and Menghao Hu and Weihuang Zhong and Tong Zhang and Shuai Wang and Yixin Zhang and Mingjie Shao and Wanli Ni },
  journal={arXiv preprint arXiv:2506.15167},
  year={ 2025 }
}