Learning To Score Olympic Events

While action recognition has been addressed extensively in the field of computer vision, action quality assessment has not been given much attention. Estimating action quality is crucial in areas such as sports and health care, while being useful in other areas like video retrieval. Unlike action recognition, which has millions of examples to learn from, the action quality datasets that are currently available are small -- typically comprised of only a few hundred samples. We develop quality assessment frameworks which use SVR, LSTM and LSTM-SVR on top of spatiotemporal features learned using 3D convolutional neural networks (C3D). We demonstrate an efficient training mechanism for action quality LSTM suitable for limited data scenarios. The proposed systems show significant improvement over existing quality assessment approaches on the task of predicting scores of Olympic events both with short-time length actions (10m platform diving) and long-time length actions (figure skating short program). While SVR based frameworks yields better results, LSTM based frameworks are more intuitive and natural for describing the action, and can be used for improvement feedback.