Early Stage Influenza Detection from Twitter

Influenza is an acute respiratory illness that occurs virtually every year and results in substantial disease, death and expense. Detection of Influenza in its earliest stage would facilitate timely action that could reduce the spread of the illness. Existing systems such as CDC and EISS which try to collect diagnosis data, are almost entirely manual, resulting in about two-week delays for clinical data acquisition. Twitter, a popular microblogging service, provides us with a perfect source for early-stage flu detection due to its real-time nature. For example, when a flu breaks out, people that catch the flu posts related tweets which enables the detection of the flu breakout promptly. In this paper, we investigate the real-time flu detection problem on Twitter data by proposing Flu Markov Network (Flu-MN): a spatio-temporal unsupervised Bayesian algorithm based on Markov Network for early stage flu detection. We model the flu dynamics as 4 different phases: a non-epidemic phase (NE), a rising-epidemic phase (RE), a stationary-epidemic phase (SE) and a declining-epidemic phase (DE) and our model tries to identify the phase-transiting at the earliest stage. We test our model on real Twitter datasets from the United States and compare our model with baselines in multiple applications, such as real-time flu breakout detection, future epidemic prediction. Experimental results show the robustness and effectiveness of our approach and that we can detect flu breakouts promptly and accurately. Further, based on Flu-MN, we build up a real time flu reporting (detecting) system that may be of help to government or health organizations in identifying flu outbreaks and facilitating timely actions to decrease unnecessary morbidity mortality.