GenerationMania: Learning to Semantically Choreograph

Beatmania is a rhythm action game where players take on the role of a DJ who performs music by pressing specific controller buttons to mix "Keysounds" (audio samples) at the correct time. Unlike other rhythm action games such as Dance Dance Revolution, players must play certain notes from up to eight different instruments. Creating game stages, called "charts," is considered a difficult and time-consuming task, and in this paper we explore approaches in computer generation for these maps. We present a deep neural network based process for automatically generating Beatmania charts for arbitrary pieces of music. Given a raw audio track of a song, we identify notes with its corresponding instrument, and use a neural network to classify each note as playable or nonplayable. The final chart is produced by mapping playable notes to controls. We achieve an F1-score on the core task of Sample Selection that significantly beats LSTM baselines.