We have made changes to the U.S Interventions Model to improve its accuracy and wanted to share details on what changed and why.
There are two ways in which we improve our model to increase its accuracy: (1) better data to feed the model and (2) underlying improvements to the model’s capabilities.
1. More and better data to feed the model
The U.S. Interventions Model runs new calculations every 24 hours using the most current available data as inputs. Because of this, you may observe regular changes in the projections as the fresh data is imported.
Generally speaking, more data and improved accuracy of data yield better projections.
2. Improving our modeling capabilities
As we learn more about the behavior of COVID, we can improve the capabilities of our model to yield more accurate projections. Because we value truth-seeking and know that many people are relying on this model to make decisions, we feel it is important to share a summary of these model design changes.
We are updating the U.S. Interventions Model to consider the hospitalization numbers reported by states.
When we began work on the U.S. Interventions Model, our estimate of COVID hospitalizations was derived by applying a hospitalization rate to a reported number of confirmed cases. At the time, most states were not reporting actual hospitalizations.
Now, most states are reporting some measure of COVID hospitalizations. The benefit of this update to the model is that the model now considers real-world hospitalization data. The downside, which we have learned anecdotally, is that some states are under-reporting COVID hospitalizations.
Under-reporting is caused by the fact that some hospitalized COVID patients have not been tested due to lack of testing capacity, and therefore are not counted in the official reports. Sometimes states report statistics like “cumulative hospitalizations” which are difficult to convert into the “current” hospital population numbers that the U.S. Interventions Model uses. Because the reported hospitalization numbers in the update are lower than the currently estimated hospitalization numbers, it may appear that the model is becoming more optimistic.
We are updating the U.S. Interventions Model’s assumptions about the number of hospitalizations, and how severe they are, based on a better understanding of the disease.
From reviewing the data, we believe that the rate of COVID hospitalization is lower than initially thought (4% now vs. 7% previously) because there are a higher number of asymptomatic infected people than previously thought. However, of the patients that are hospitalized, the severity of the hospitalizations, that is, the rate at which they require ICU or ICU-type treatment, is much higher than initially thought. In addition, these patients are staying at the hospital longer, taking up ICU resources for more time than previously estimated (14 days now vs. 6-10 days previously).
ICU-style treatment capacity, where it does exist, tends to be small and hard to increase. Because of this fact, the increased rate of case severity means that the scenario of ICU-overload is likely to cause a greater increase in death rates. This change to the model may cause it to appear more optimistic if we remain within hospital capacity, and more pessimistic if we exceed it.
We believe in being open, transparent, and truth-seeking in our approach to improving the model as we learn more about COVID daily.
Again, it is important to remember that the model responds to new data, such as updated hospitalization counts, every day. These changes are natural and expected.
When we make meaningful design changes to the model, we’ll summarize them on this blog and update our documentation accordingly. We are committed to regularly updating our community of users as our thinking, data, and tools evolve and improve.