UMass and Research Partners Aim to Improve Flu-season Forecasts

AMHERST — Research teams, including one led by biostatistician Nicholas Reich at UMass Amherst, are participating in a national influenza-forecasting challenge to try to predict the onset, progress, and peaks of regional flu outbreaks to aid prevention and control. This year, the Reich Lab is leading an effort to improve the forecasting by increasing the collaboration between groups.

“Every year, the Centers for Disease Control host a flu-forecasting challenge,” Reich said. “It’s the most organized and public effort at forecasting any infectious disease anywhere in the world. Our lab is now in our third year of participating, and we find that each year we get a little better and learn a bit more.

“This year, we wanted to take it to the next level, so we worked with other teams year-round to develop a way that our models could work together to make a single best forecast for influenza,” he went on. “This entire effort is public, so anyone can go to the website and see the forecasts.”

While this flu season has started earlier than usual in the northeastern and southern regions of the U.S., according to the most recent data, the forecasts are still showing a fair amount of uncertainty about how big a season it will be, Reich said. “The holiday season is a notoriously difficult time to forecast because typically fewer people go to the doctor, and yet everyone is traveling around spreading or being exposed to infections such as flu.”

Reich and colleagues at UMass Amherst’s School of Public Health and Health Sciences collaborate with teams at Carnegie Mellon University, Columbia University, and a group at Los Alamos National Laboratory in New Mexico, in a group they have dubbed the FluSight Network. It issues a new flu season forecast every Monday for public-health researchers and practitioners that compares the flu trajectory this year to past years.

In a recent publication, Reich and colleagues state that their aim is to “combine forecasting models for seasonal influenza in the U.S. to create a single ensemble forecast. The central question is, can we provide better information to decision makers by combining forecasting models and, specifically, by using past performance of the models to inform the ensemble approach.”

Added Reich, “we are working closely with our collaborators at the CDC to determine how to improve the timeliness and relevance of our forecasts.”

To prepare for this flu season, he and colleagues spent many hours designing a standard structure that each team needed to use when submitting models. This allowed for comparison of methods over the past seven years of flu data in the U.S. They also conducted a cross-validation study of data from the past seven flu seasons to compare five different methods for combining models into a single ensemble forecast. They found that four of their collaborative ensemble methods had higher average scores than any of the individual models.

The team is now submitting forecasts from their best-performing model and are posting them once a week this season to the CDC’s 2017-18 FluSight Challenge. Reich estimates there are about 20 teams this year participating in the CDC challenge nationwide, who produce about 30 different models. Each model forecasts the onset of the flu season, how it will progress over the coming few weeks, when it will peak, and how intense the peak will be compared to other seasons.

In a heavy flu season, between 5{06cf2b9696b159f874511d23dbc893eb1ac83014175ed30550cfff22781411e5} and 12{06cf2b9696b159f874511d23dbc893eb1ac83014175ed30550cfff22781411e5} of doctor’s visits are for influenza-like illness, and that number varies regionally in the U.S. This metric is one of the key indicators for the CDC of how bad the flu season is, and it is the measure used in the forecasting challenges.

“Certainly for the CDC, there are policy decisions that could be impacted by these forecasts, including the timing of public communication about flu season starting and when to get vaccinated. Models can help with all of that,” Reich said. “Also, hospitals often try to have enhanced precautions in place during a certain peak period for the disease. If you do that too early, or for too long, you run the risk of individuals getting tired of taking the extra time to comply with the policies.”

Hospital epidemiologists and others responsible for public-health decisions do not declare the onset of flu season lightly, he noted. In hospitals, flu onset — a technical set of symptoms reported to physicians — triggers many extra time-consuming and costly precautions and procedures such as added gloves, masks, and gowns; donning and doffing time; special decontamination procedures; increased surveillance; and reduced visitor access, for example. There is also healthcare worker fatigue to consider. Hospitals want to be as effective and efficient as possible in their preparations and response to reduce time and money spent and worker burnout.

The public-health effort to improve flu season forecasts is relatively recent, Reich said. “There has been tremendous progress in how we think about infectious disease forecasting in just the last five years. If you compare that to something like weather forecasting, which has been going on for decades, we’re in the middle of a long process of learning and improvement. Someday, we might be able to imagine having a flu forecast on our smartphones that tells us, for example, it’s an early season and I’d better get Mom to the clinic to get her vaccination early this year. We’re close, but that’s not here quite yet.”