Month: April 2020

Given the recently announced antibody tests from NY, I wanted to take a closer look at my model and compare the results.  It is important to be cautious about overinterpreting preliminary results like this; it is using a new test, for a new disease, on a sample for which we do not know how well it represents the population, but it is also the best test we have so far.

In my simple model for daily death, there are 2 inputs: speed of transmission, R0, and mortality rate.  Given that we only know the output, we cannot fully determine the value of both inputs, however to fit the empirical evidence it must be that if the mortality rate is lower then the speed of transmission must be higher and vice versa.  In other words, each mortality rate has an associated path of transmission and thus a path which determines the total number of infections/deaths.

The model can tell us which combinations of transmission and mortality are possible but not what either of them is.  To do that we need to look at other data.  In my first post on mortality, I focused on studies that estimated mortality rate directly, from which we then estimate the transmission rate.  The other approach is to focus on the number of infections to give a rate of transmission and then see what that implies for mortality rate.

When looking directly at studies on mortality, I concluded that 1% was my best estimate.  The New York test is the other approach, looking at the number of infections.  We can now look at what a completely different data set tells us.  I have drawn 3 paths for NY infections below, all of which can fit the basic data we have seen on death rates.

There is clearly a non-linear relationship between % mortality and % infections in the population.  If I take 1% as the base case, then a rate of 0.5% would require a far higher infection rate while a rate of 1.5% instead would not change the number of infected people nearly as much.

My model predicts a mortality rate of 1% and current deaths data, then the population infection rate would be 12% of the NY State population.  This is remarkably close to the estimate from the NY test result of just under 14%.

The NY test could imply mortality rate a little below 1%, but on the other hand we know there is material underreporting of COVID deaths and so, one could equally argue that the mortality rate is a little over 1%, consistent with the same data.  Some media outlets are using the results to suggest the mortality rate is 0.5%, a poor estimate and a bad way to use statistics.  This ignores the important lag between infections and death as well as the underreporting of deaths.

My initial estimate of the mortality rate was around 1% which is confirmed by this test.  This makes all the projections for future waves of infection more likely to be ones we should be confident about.

Is this a surprise?

To me, no.  It is very much in the range of where medical expert consensus had it beforehand.  I do not think I have been proposing a model any different from mainstream medical thought, but I have found it useful to be able to make my own projections and explain the implications.

However, looking at media reports and my Bloomberg inbox, I have only read people saying that this data shows that the mortality rate is lower than expected. Some are very excitedly saying is a “game changer” and “most of our clients think this is a potentially important development”.  If this reaction is common, then it shows that while the news is virtually only about COVID, very little of it is genuinely informative and people’s understanding of the data is extremely poor.

One of the persistent misrepresentations of the data is that the reported case fatality rate is an estimate.  It is not.  It is a statistic.  It is simply confirmed deaths divided by confirmed cases.  Given the low levels of testing, we know that reported cases are a small fraction of actual infections.  Hence it may be used as an estimate, but it is a truly terrible estimate if you simply take the headline number.  Did anyone really think that mortality was “estimated” to be 13% in the UK for people who got infected?   Only if you thought this was an estimate would this “new” estimate of 1% seem lower to you.

Policy implications

The debate on deaths vs economic costs of reopening will continue, but I hope that so many pseudo-experts will now refrain from saying that COVID has a mortality rate close to flu.

Using the estimate of mortality rate from the NY data to imply infection rates in other countries, suggests that herd immunity is a very long way off.  The UK would have infection levels of just 4%.

PS Treatment news

Yesterday the S&P dropped 1.5% on news that Gilead’s drug which they touted as a treatment for COVID-19 had failed its first clinical trial.  It was a major positive news story a week ago, which tells us how people struggle to interpret news and statistics and the media struggles to provide sensible coverage.  As I mentioned on April 20^th “It is overwhelmingly likely that it is not true … Until we have some proper testing, I will assume this is a nonsense story”

Similarly, please do not inject or drink disinfectant.  Killing a virus outside your body is not the same as killing it inside your body.  On the bright side, this was a Trump theory that was so bad even Fox News has not been pushing it.

This data point seemed important to me and worth a short immediate note.   We have just had released the reasonably sized antibody test from an area of high infection.

The rate of infection in New York is estimated to be 14%.

This implies around 2.7m people in NY State have had the virus.

A little over 20,000 are reported to have died from COVID.

This gives a mortality rate of 0.8%

We know that the number of deaths is likely to be underestimated due to people dying at home or never being tested, so the actual mortality rate would be higher than this.

Perhaps 1% is still the best estimate.

This data confirms the studies from around the world and means that the models we have been working with look to have the correct parameters.  It makes the idea that the mortality rate is far lower even more implausible.

As the world heads towards easing of lockdowns, it is a good time to look again at the data.  The tone I read in most media outlets, and certainly from the markets, is that the peak has passed and we will soon be able to return to something like normality.  When I look at the data however, I struggle to reach the same conclusion.

One key metric is the pace of decline in daily deaths in countries/states which have been pursuing similarly strict lockdowns. Many of these have already eased some restrictions, or are actively planning to do so over the next few weeks.

If we turn the data into an % index series to adjust for different sizes of area/population and level of infection, we see a similar picture of slow declines.

Data notes:  Source is Bloomberg. I have taken a 3-day moving average as daily data is very noisy in some countries.  I have smoothed the mid-April spike in NY, due to major changes in data collection.  I have also lagged the UK data by a week, as we imposed lockdown later.

Whilst we are past the peak of this wave of infection, it has taken extreme personal and economic restrictions to bring the virus transmission rate to an R0 marginally below 1.

Recent data from New York has been more encouraging, also tallying with information from the UK government that London data is improving more rapidly with some areas of the UK still getting worse.  Last night, Chris Whitty, Chief Medical Officer for England, explained that we still do not know which lockdown measures really matter for slowing infection rates.  This makes it very hard to partially ease restrictions with confidence.

The situation in the US overall looks are even more concerning.  Here I exclude New York and New Jersey as their very high rates overwhelm the picture elsewhere.

At best, we could argue that we are at peak and that rates will decline from here, but it is premature to have much confidence in that. Yet the US is already materially reopening some States and is under pressure from Trump to open even more very rapidly.

If we look at state level data, most tell a similar story of stabilisation at best. Given the lags, we may see rates decline soon but we should then expect a rapid re-acceleration 3 weeks after lockdowns are relaxed.  The absolute numbers are still low in most states, with only a small proportion of people being infected.  This is perhaps why it is still possible to think that the problems are small and politicians can attempt to say they are under control.  But we have seen over and over again, how with exponential growth we rapidly move from 50 deaths a day to 1,000 deaths a day

Not all US states are even as good as this; despite the “stay at home order”, we are still seeing an escalating problem.

Conclusion

The evidence suggests that in areas that enacted strict lockdowns, we are indeed past the first peak.  However, lockdowns have not necessarily been as effective as hoped in rapidly bringing down infection rates.  Outside worst hit areas, compliance may be far poorer.

An early relaxation of the current restrictions will mean limited respite for health services, and if infection rates rise again they will do so from an already elevated level.  By the time we know the impact of policy changes, it will be too late to stop the next peak from being far higher than the last.

I have had some feedback from readers on mortality rate and why I have paid so little attention to it so far.

Why is the stated mortality rate around the world so different?

UK and Italy are on the high side with mortality rate around 13%.  On the low side, Germany at 3% and Norway at 2.3%.  The US is in between at 5%.  There are various issues with data collection but the biggest factor is how widespread testing is.

In relatively overwhelmed countries e.g. Italy and the UK there has been a limited roll out of testing, with a high proportion of those tested because they have symptoms already.  In the extreme case, if we only tested people for COVID who had already died with COVID symptoms, then it would have a mortality rate of approaching 100%.   In the UK a quarter of people tested test positive, which means we are far from seeing a random sample.

The countries with much high testing rates will give a much better idea of the actual mortality rate.  Norway has only 5% of tests come back positive and their mortality rate is much lower.  It does not necessarily mean that Norwegian people have better outcomes, it is just that they are better at catching the cases where people have COVID but are either asymptomatic or recover by themselves at home.

What is the mortality rate really?

Getting a narrow range on this estimate is not at all easy.  We really need random sampling and tracking of outcomes.  This has recently begun in Germany, but it will be a long time before we get decent results.  While it is known there is some underreporting in the number of deaths e.g. care homes, it is also clear that underreporting of cases from incomplete sampling/too few tests is a far bigger distortion.  This means that the mortality rate will be lower than the reported mortality rates at the moment.

The lowest estimate from a good source that I have come across is from the Diamond Princess cruise ship; adjusting for the average age of the passengers gives an estimate of 0.66%
https://www.eurosurveillance.org/content/10.2807/1560-7917.ES.2020.25.12.2000256

I have used 1% as my base case in my modelling, but 0.5% would also seem very reasonable.

Could the mortality rate be much lower?

There are some scientists arguing that the mortality rate is far lower.  Their argument assumes the virus was spreading earlier than we thought and it must be far more rapidly transmitted.   It follows that lots of us have been infected without knowing it and therefore the mortality rate is extremely low.

A recent highly publicised study in Santa Clara used antibodies tests to find that 50 times as many people were infected than we thought with 50 out of 3300 testing positive for antibodies associated with COVID.  https://www.nature.com/articles/d41586-020-01095-0

I find it very hard to reconcile these theories with the known data.  Countries that have done large scale testing such as Iceland have not found the large numbers of positives that would be consistent.

If the virus had infected 50 times as many people as we thought, then virtually all cases must not show any symptoms.  This is not consistent with other reliable studies.  Studies with a controlled set of patients such as Diamond Princess do show that perhaps half of cases are asymptomatic, which is why the mortality rate of people infected could be below 1%.  There is no evidence that a large majority of infected people are asymptomatic.

False positives

The more likely explanation for the Santa Clara result are false positives which occur in all medical tests.  The test for HIV is also an antibodies test i.e. we do not detect the virus directly but we can detect the antibodies which are associated with our immune response to the virus.  If 3300 people were tested for HIV using a rapid test, we would EXPECT to see 50 positive tests even if NO ONE has HIV.  This is because the test has a 1.5% false positive rate.

Can a treatment breakthrough reduce the mortality rate?

Another argument for a lower mortality rate is that we will develop treatments for the disease.  The stock market soared on “news” that Gilead had developed a treatment https://www.thetimes.co.uk/article/coronavirus-patients-given-us-trial-drug-are-off-ventilators-in-a-day-pb3jnzf3k

I see two problems with this

1. It is overwhelmingly likely that it is not true. I have written before at length about the misuse of statistics and this is a classic example.  Every possible drug is being given to patients with a view to treating Covid, but these are not the controlled trials that would confirm effectiveness.  By random chance, some outcomes will be better than others.  If I dressed all patients in different coloured gowns, I would find that those wearing perhaps pink gowns had better outcomes than those wearing red ones.  Swapping all gowns to be pink would not then lead to lower mortality rates. Until we have some proper testing, I will assume this is a nonsense story.
   
   
2. Even if it turns out to be true, this treatment only works for people on ventilators. It is only relevant for bringing down the mortality rate in a scenario where everyone has access to a ventilator i.e. the number of people infected is small. It does not change at all the need for lockdown or have any major economic implications as the story currently reads.  For a treatment to have an impact on lockdown policy, it would need to have a material impact on patients to stop them requiring hospitalisation.

Could the mortality rate be much higher?

If treatment makes a large difference to the outcomes, here there could be an important skew to the mortality rate higher.  Currently, the countries with the lowest mortality rates have, in common, a low number of cases and excellent access to healthcare.  There is good evidence so far that early treatment and access to oxygen makes a big difference to the outcomes.  If a patient is admitted late to ICU, then the prognosis is very poor.  With Boris Johnson for example, early intervention with oxygen led to a positive outcome.  For many this is not the case and if our health systems are overwhelmed then very few will get this level of treatment and the mortality rate could be far higher. My fear is that if this virus gets out of control, then the mortality rate will be far higher than we have observed so far with total deaths far higher even than the current high estimates.

If it is much lower what difference does this make to lockdown policy?

The reason that I have focused on R0 rather than mortality rate is that it is far more important for policy decisions.  A lower mortality rate is relevant in two cases:

1. We aim for herd immunity.
   This was the initial idea for the UK and requires a lower mortality rate that we have estimated, and thus the number of people already infected to be far higher.  With current estimates, we only get herd immunity after perhaps 500,000 people die in the UK, which dramatically swamps the system.  The other big problem with this is there is only limited evidence on the level of immunity we get after recovering from COVID.  Looking at our response to other respiratory viruses, the current best guess is that it provides a decent level of protection for only a year or two.
   
   
2. We let people die
   The idea here is that “losses” to the virus are acceptably low in comparison to the economic damage we otherwise take.  This is the common argument of the right and, particularly in the US.  I will leave it up to each reader’s own ethics on how they would make that trade-off.  But I will reiterate that we currently have no evidence that the mortality rate would be lower than 0.5% i.e. 1 million US deaths, and good reason to think it would be a multiple of that with this many cases and thus no medical treatment available.