Data and analytics have an undeniable power to inform and influence our decisions and actions. Their underlying complex processes are often distilled into a few key metrics that are delivered via dashboards with interactive visualizations. But you should never take at face value what those metrics actually represent, and on what data they were based. In other words, before you start looking at the data, understand the metadata – that is, the terms behind the metrics.

The current coronavirus pandemic reminds us that the terms used to describe the metrics matter. So let’s examine some of the terms associated with coronavirus dashboards.

Four key terms

• Confirmed cases — Not to be confused with total cases, which would require the impractical widespread, comprehensive and recurring testing of an entire population. A confirmed case is the result of a person testing positive for coronavirus.
• Viral versus antibody tests — A viral test tells you if you have a current infection. An antibody test tells you if you had a previous infection. This distinction is important to note for two reasons.
  • First, a positive result from a viral test is a negative thing (you're infected now and may be infectious to others). But a positive result from an antibody test is a positive thing (you're no longer infected or infectious to others).
  • Second, a viral-test negative only means you were not infected when you were tested – it does not mean you are immune. For example, I viral-tested negative in mid-May, but I could still get infected. Keep in mind that medical scientists are not sure if antibody-testing positive indicates immunity.
• Case fatality rate (CFR) — The ratio of the number of confirmed deaths and the number of confirmed cases. This should not be confused with infection fatality risk (IFR), which is the probability an infection results in death. The accuracy of CFR is impacted by some infected persons being either asymptomatic or not developing symptoms severe enough to seek medical attention. These people either don’t attempt to get tested, or are denied testing because they’re not “sick enough” to test. The latter varies by location. Where I live, you must be preapproved to get tested outside of a hospital emergency room. Paradoxically, you often cannot get preapproved unless you're sick enough to go to a hospital emergency room.
• Recovered cases — The number of confirmed cases that did not tragically result in death. However, some of the recovered remain non-terminally ill for a while and a few show signs of potentially chronic non-terminal illness. This possible long-term healthcare crisis isn’t getting much attention.

Two "trending" terms

As the availability and accessibility of testing significantly increased, so did the number of confirmed cases. Those of us in the data and analytics industry were not surprised to see a common shift in dashboards and metrics at this point. As expected, more focus was placed on rolling daily averages than individual daily totals as trending was emphasized to measure whether the pandemic was getting better or worse. Subsequently, two other trending metrics became highlighted more frequently:

• Positivity rate — The percentage of the total number of viral tests that yielded a positive result. If it’s high, or trending up, it signals a possible increase in coronavirus community spread. If it’s low, or trending down, it signals the possibility the coronavirus outbreak is slowing. Note that positivity rate is a better indicator than confirmed cases – but only if a widespread number of daily tests continue to be performed.
• Hospitalization rate — The percentage of people who viral-tested positive and also required overnight hospitalization. This metric is often shown alongside hospital capacity and intensive care unit (ICU) capacity, including the percentage of those capacities currently consumed by coronavirus patients.

Looking beyond the metrics – two more concepts

Beyond the metrics themselves, there has also been an increased emphasis on the following fundamental concepts:

• Correlation — It’s almost always best to not look at any metric in isolation. The correlation between positivity rate and hospitalization rate, for example, attempts to quantify how many infected people are getting more than mildly sick, which is probably the best measure of the pandemic’s severity. Public health experts also caution metrics can be misleading if not correlated with context-specific underlying data. These correlations include:
  • The time, location and duration of exposure.
  • The government, social and individual responses to the pandemic.
  • An individual’s ability to recover based on age and preexisting health conditions.
• Lag — There’s a lot of lag in coronavirus metrics. It can take up to 2 weeks after exposure for you to develop symptoms indicating a likely infection (this is why individuals who may have been exposed are quarantined for 14 days). Therefore, a confirmed case reported today is likely associated with an exposure event from at least 2 weeks ago (if you got tested as soon as you showed symptoms). And it can take another 2 weeks (in mild cases) to 4 or up to 6 weeks (in severe-critical cases) to either recover or die. Which means coronavirus deaths lag behind infections by 1 to 2 months. This explains why deaths can be trending down while the pandemic is worsening (as was the case in the US as of late June). It’s also why you cannot immediately assess the impact of a decision (e.g., reopening closed bars) or an action (e.g., holding indoor political rallies) on coronavirus infection risk. Because you often won’t know the impacts until at least a month later. Lag is also one of the reasons contact tracing is so important. Due to lag, the time and location of an individual infection is rarely obvious.

The value of data-driven decisions

Certainly, dashboards and metrics enable data-informed decisions. And most leaders at the forefront of the coronavirus response are rightly advocating for this. Coronavirus reminds us that dashboards are amazing tools – as long as we know what the terms behind the metrics actually mean.