Bayesian Optimization for Iterative Learning

Deep (reinforcement) learning systems are sensitive to hyperparameters which are notoriously expensive to tune, typically requiring running iterative processes over multiple epochs or episodes. Traditional tuning approaches only consider the final performance of a hyperparameter, ignoring intermediate information from the learning curve. In this paper, we present a Bayesian optimization approach which exploits the iterative structure of learning algorithms for efficient hyperparameter tuning. First, we transform each training curve into numeric scores representing training success as well as stability. Second, we selectively augment the data using the information from the curve. This augmentation step enables modeling efficiency. We demonstrate the efficiency of our algorithm by tuning hyperparameters for the training of deep reinforcement learning agents and convolutional neural networks. Our algorithm outperforms all existing baselines in identifying optimal hyperparameters in minimal time.