Note: This blog post is a higher-level description of Covid Act Now’s four COVID metrics. For a more technical, “kitchen sink” description of assumptions and methodology, please see our references and assumptions document.
Fighting COVID is hard, but understanding where we stand against COVID just got easier.
A policy consensus is emerging how to reopen safely and several metrics are key to understand when and how.
The four key metrics are:
COVID case growth: Are the number of cases increasing, stabilizing, or decreasing?
Testing capacity: Are we testing broadly enough to track disease spread?
ICU safety margin: Is there enough ICU hospital capacity to absorb a possible wave of new COVID hospitalizations?
Contact tracing: Are we finding and isolating new cases before COVID spreads?
As of today, we track these four metrics for all 50 states and 93 of the most populous counties in the United States. An additional 1,727 counties have at least one metric, and more counties will be added to the tally in the coming days.
Why We Built The Dashboard
The dashboard is based on feedback from federal, state, and local decision-makers. The feedback was clear: without actionable data, it is incredibly challenging to know whether, when, and how to reopen.
Our goal building the dashboard was to solve this problem by providing easy-to-understand, actionable, real-time metrics relevant to COVID.
Data-driven decisions help us better manage our response to COVID. The better we manage our response to COVID, the less our economy will be harmed and the more lives we can help save.
In any given state or county, cases of COVID can be increasing, stabilizing, stable, or decreasing. We color the number red if the cases are increasing (R(t) > 1.4), orange if the cases are stabilizing (R(t) 1.1 – 1.4), yellow if the cases are stable (R(t) 0.9-1.1), and green if cases are decreasing (R(t) < 0.9).
In order for a state to reopen safely, the number of COVID cases should be clearly decreasing. In other words, R(t), the important epidemiological metric that we described previously, should be less than .90
For example, above is a chart of the infection growth rate in Montana (as of May 12). The R(t) of 0.96 means that each person with COVID is, on average, infecting 0.96 other people. Therefore, the total number of cases is (at least in the present moment) decreasing.
Because of how we weight our data (Gaussian smoothing), and because of potential reporting delays and errors in the incoming case data, we need 10 preceding days of data before we can calculate a final R(t) value. Therefore, we notate preliminary R(t) values — values for which we don’t yet have 10 preceding days of data — with the dotted line.
To calculate R(t), we use new positive cases and COVID death data sourced from The New York Times.
An emerging policy consensus suggests that states should wait to reopen until cases have steadily decreased (an R(t) less than 1.0) for at least two consecutive weeks. Johns Hopkins University and American Enterprise Institute both include two weeks of decreasing cases as a metric for reopening in their policy guidelines, as do (as of 12 May) fourteen states (CN, DE, IN, NV, NY, RI, SD, WI, VA, KY, KS, HI, DC, and CO.). The Center for American Progress also calls for a sustained decline in positive cases in its reopening recommendations, and Duke University’s reopening plan cites AEI’s two-week figure.
In order to consider opening up, states and counties should be testing a sufficient percentage of the population to ensure that there are no undetected infection clusters that could lead to a second wave of disease.
How do you know if you’re testing enough? One way to measure testing capacity is to look at the percentage of positive tests. As Michael Ryan, executive director of the WHO Health Emergencies Program, puts it, “If 80-90% of the people test positive, you are probably missing a lot of cases.” The White House Coronavirus Task Force’s reopening guidelines also point to a reduction of positive tests as a percentage of total tests as a key metric for reopening.
So, low is good. How low is low enough?
We have created four categories for the percentage of positive tests: above 20% is critical (in red), between 10% and 20% is high (in orange), between 3% and 10% is medium (in yellow), and below 3% is low (in green).
How did we arrive at those standards? 10% — the threshold between high and medium — is the World Health Organization’s recommendation for maximum test positivity (i.e., you want to be under 10%). Harvard University epidemiologist William Hanague and Admiral Brett Giroir, a member of the White House Coronavirus Task Force, have also both cited the 10% figure. 20%, the threshold between critical and high, is twice that number.
The 3% cutoff between medium and low is roughly the rate achieved by the countries successfully containing COVID. South Korea, for instance, has a test positivity rate of 1.6% (as of 12 May).
ICU Safety Margin
If or when there’s a new wave of COVID infections, are our hospitals ready to absorb a possible surge of patients?
This question is answered by our third metric: COVID ICU capacity. The White House’s reopening plan, AEI’s plan, and Harvard University’s Safra Center for Ethic’s plan all cite health system capacity as an essential metric for reopening. But how do we measure it?
It’s a bit wonky, so let us explain by example:
Here is a chart of ICU headroom in New York as of June 4. New York has 4,121 ICU beds. We estimate that 1359 are occupied. We make that calculation by taking the typical number of occupied beds and applying a 30% “decompensation factor” to account for deferral of elective procedures and/or surged ICU capacity.
Of the remaining 2,762 ICU beds, we estimate that 1,037 are occupied by COVID cases, which amounts to 38% of ICU headroom.
The 38% statistic is indicative of the healthcare system’s ability to absorb a surge of new COVID hospitalizations, should a new wave of infections occur. This metric is very much an approximation of hospital and ICU surge capacity; it just gives a rough sense.
We use 50% available ICU headroom as the threshold between low and medium per Resolve to Save Live’s policy recommendation. They suggest that in order for states to consider reopening, COVID cases in the ICU should be able to double without overwhelming hospital systems.
When people contract the virus, they do not show symptoms right away. Even as states begin to reopen, people will need to quarantine themselves if they have been silently exposed to someone with COVID.
How will they know? That’s where contact tracing comes in.
Because of this problem, it is critical that enough tracing capacity exists to rapidly trace the contacts of individuals who test positive for COVID. Those contacts can be tested, quarantined (if necessary), and asked about whom else they have come into contact with. Because exposed individuals begin infecting others, it is critical that this process be completed in less than 48 hours. If this routine of testing and tracing is done quickly and completely, it can contain COVID, as we have seen in South Korea and Taiwan, and without the need for costly lockdowns.
The White House Coronavirus Task Force’s Guidelines say that contact tracing is a “core responsibility” of states in order to be prepared to reopen. As of May 21, 27 states (CA, CT, DE, FL, HA, IL, IN, KS, KY, ME, MD, MI, MO, MT, NE, NV, NM, NY, NC, ND, OH, PA, RI, SD, WA, WV, and WI) call for contact tracing in their reopening plans. The American Enterprise Institute’s roadmap to reopening says states must massively scale up contact tracing and isolation/quarantine of traced contacts.
So how do we calculate contact tracing capacity? Experts recommend tracing contacts of someone who tests positive for COVID within 24 hours, to contain the potential of transmission. Based on conversations with practitioners and public health experts, our metric assumes that tracing all contacts for each new positive COVID case requires an average of five full-time contact tracers.
Therefore, our contact tracing metric measures the percentage of new cases for which all contacts can be traced within 48 hours relative to available contact tracing staff in the state (assuming 1:5 new-positive-COVID-case:contact-tracing-staff ratio).
We use green if greater than 90% of the contacts can be traced within 48 hours, yellow if between 20% and 90% of the contacts can be traced within 48 hours, orange if between 7% and 20% of the contacts can be traced within 48 hours, and red if fewer than 7% of the contacts can be traced within 48 hours.
Here is an example from Wyoming:
As of June 13, Wyoming has an average of 12 new cases per day. If Wyoming needs 5 contact tracers per case, that would be 60 contact tracers necessary to trace all cases in 48 hours. Since Wyoming has 50 contact tracers, that is enough to trace 82% of cases.
How did we choose our targets? Research estimates that the infection rate can be driven below 1.0 if 70-90% of cases are identified and 70-90% of those contacts are traced and isolated within 48 hours or less. Therefore, we chose 90% as the cut-off between green and yellow.
The boundaries between yellow, orange, and red are trickier. When less than 90% of positive cases have their contacts traced within 48 hours, contact tracing will likely be insufficient to contain COVID. We use 90% as the boundary between green and yellow. In the absence of expert consensus, we have set inclusive lower thresholds for yellow and orange. We peg the cut-off between yellow and orange at 20% — the number required for there to be one contact tracer per active case per 48 hours — and the cut-off between orange and red at 7%. Every state currently coded red is either currently experiencing a new outbreak or effectively has no tracing capacity.
A state can become green either by increasing the number of contact tracers, or by decreasing the number of new daily COVID infections.
Are These Four Metrics Enough to Assess Reopening?
These four metrics are not the be-all, end-all that tells us whether a state or county is able to safely reopen. It’s a complicated case-by-case decision and includes other factors such as PPE availability, ability to trace positive cases, and much, much more.
Nevertheless, these four metrics provide important benchmarks as we figure out how to reopen our country safely.
Our hope is to create a common, data-driven language for discussing our COVID response and to provide a foundation that healthcare and policy decision-makers can build on.