Copyright 2020 Robert Clark
Attached below is an email message to the authors of a recent report on hydroxychloroquine and COVID-19, “Observational Study of Hydroxychloroquine in Hospitalized Patients with COVID-19.” In the first part, I discuss a simplified approach to compensating for the fact that in non-randomized trials you often don't have comparable numbers in the treatment groups compared to the control groups. The method commonly used to compensate for this, "propensity score matching", is computationally complex and often leaves doubt about the validity of the calculations. The method I suggest is simple and can be done by anyone with a hand-calculator.
In the second part, I discuss a surprising calculation from the data. The authors had concluded that HCQ offered no benefit. But when I calculated the survivability numbers for intubated patients, those on ventilators, I was startled to find the ones on HCQ had twice as good survivability than those not on HCQ! This is an extremely important point because in general the survivability of patients on ventilators is so poor. In fact, in New York only 20% survive being on ventilators with COVID-19. It's mystifying that the authors did not find this significant enough to mention. If you have a medication that can double the chance of survival of patients on ventilators then that fact should be heralded rather than hidden.
Lastly, the authors have suggested in interviews that their results show HCQ is no better than "standard care". But it was not mentioned that in this study, and in actually in several other studies, what "standard care" means is that the patients are just given other antivirals, and one of them in this study is the potent antiviral and anticancer medication interferon. Just calling the control group under "standard care" would mistakenly give readers the impression that it is perhaps just bed rest and extra oxygen. Then what this study really shows is not that HCQ is ineffective but rather that it is no more effective than those other antivirals. And in actuality, that is also doubtful because those other antivirals did not have the good effect HCQ had for patients on ventilators.
Robert Clark
UPDATE, 6/26/2020:
Recent news releases from the RECOVERY trial suggest that hydroxychloroquine offers no added benefit in COVID-19 treatment, but that dexamethasone improves survival 30% for COVID-19 patients on ventilators.
It is easy to imagine that a drug for various reasons may be more effective for patients with different risk factors. For example, HCQ is an anti inflammatory, as is dexamethasone, so it might be expected to help intubated patients. It is notable that CT imaging can show lung damage due to COVID-19. Then whether or not HCQ does have this effect of reducing lung inflammation for COVID-19 patients might be something observable in real-time on CT scan.
This would be extremely important for determining its effectiveness in such cases since it could be seen right away if it were working or not, rather than waiting to see if the patient survived or not, which could be due to other factors anyway.
The reason why the increase in survival with HCQ for ventilated patients wasn’t seen in the overall numbers in that NEJM report was because the ventilated patients were only a small part of the total, approx. 10%, so the ventilated numbers were swamped by the larger numbers of the total cases.
At least one other study has shown that HCQ improves the survival for ventilated patients:
Research Paper
Published: 15 May 2020
Low dose of hydroxychloroquine reduces fatality of critically ill patients with COVID-19
Bo Yu, Chenze Li, Peng Chen, Ning Zhou, Luyun Wang, Jia Li, Hualiang Jiang & Dao-Wen Wang
Science China Life Sciences (2020)
Abstract
...
These data demonstrate that addition of HCQ on top of the basic treatments is highly effective in reducing the fatality of critically ill patients of COVID-19 through attenuation of inflammatory cytokine storm. Therefore, HCQ should be prescribed as a part of treatment for critically ill COVID-19 patients, with possible outcome of saving lives.
hydroxychloroquine, IL-6, mortalities, COVID-19
https://link.springer.com/article/10.1007/s11427-020-1732-2
Quite notably the authors observed in real time a reduction in the level of IL-6, a key marker for inflammation, during the treatment with HCQ. So from both CT imaging and inflammation marker readings we could tell if HCQ or other anti-inflammatories were having a beneficial effect on patients under severe disease.
Also, some other anti-inflammatories such as tocilizumab and methylprednisolone have shown benefits for treating patients under severe disease. This gives further support for the idea that HCQ as an anti-inflammatory should have beneficial effects. For this reason, reports that had shown HCQ or other anti-inflammatories ineffective overall on reducing mortality for patients under severe disease should have the data reviewed to see if specifically patients on ventilators showed increased survival on such anti-inflammatories.
UPDATE, 6/26/2020:
Recent news releases from the RECOVERY trial suggest that hydroxychloroquine offers no added benefit in COVID-19 treatment, but that dexamethasone improves survival 30% for COVID-19 patients on ventilators.
It is easy to imagine that a drug for various reasons may be more effective for patients with different risk factors. For example, HCQ is an anti inflammatory, as is dexamethasone, so it might be expected to help intubated patients. It is notable that CT imaging can show lung damage due to COVID-19. Then whether or not HCQ does have this effect of reducing lung inflammation for COVID-19 patients might be something observable in real-time on CT scan.
This would be extremely important for determining its effectiveness in such cases since it could be seen right away if it were working or not, rather than waiting to see if the patient survived or not, which could be due to other factors anyway.
The reason why the increase in survival with HCQ for ventilated patients wasn’t seen in the overall numbers in that NEJM report was because the ventilated patients were only a small part of the total, approx. 10%, so the ventilated numbers were swamped by the larger numbers of the total cases.
At least one other study has shown that HCQ improves the survival for ventilated patients:
Research Paper
Published: 15 May 2020
Low dose of hydroxychloroquine reduces fatality of critically ill patients with COVID-19
Bo Yu, Chenze Li, Peng Chen, Ning Zhou, Luyun Wang, Jia Li, Hualiang Jiang & Dao-Wen Wang
Science China Life Sciences (2020)
Abstract
...
These data demonstrate that addition of HCQ on top of the basic treatments is highly effective in reducing the fatality of critically ill patients of COVID-19 through attenuation of inflammatory cytokine storm. Therefore, HCQ should be prescribed as a part of treatment for critically ill COVID-19 patients, with possible outcome of saving lives.
hydroxychloroquine, IL-6, mortalities, COVID-19
https://link.springer.com/article/10.1007/s11427-020-1732-2
Quite notably the authors observed in real time a reduction in the level of IL-6, a key marker for inflammation, during the treatment with HCQ. So from both CT imaging and inflammation marker readings we could tell if HCQ or other anti-inflammatories were having a beneficial effect on patients under severe disease.
Also, some other anti-inflammatories such as tocilizumab and methylprednisolone have shown benefits for treating patients under severe disease. This gives further support for the idea that HCQ as an anti-inflammatory should have beneficial effects. For this reason, reports that had shown HCQ or other anti-inflammatories ineffective overall on reducing mortality for patients under severe disease should have the data reviewed to see if specifically patients on ventilators showed increased survival on such anti-inflammatories.
_________________________________________________________________________
Robert G Clark
Tue 5/19/2020 6:00 PM
Hello. I was very interested to read your article, “Observational Study of Hydroxychloroquine in Hospitalized Patients with Covid-19”. I have a few questions I hoped you or someone on your team could answer.
I.) Propensity Score Matching.
In reference to your Table 1 (attached), how do the “propensities” go into the calculation of the likelihood of death? For hypertension, when expanding the non-HCQ no. from 38 to 146 for propensity, do you keep the same ratio of deaths to the total in the category?
For example, if originally, the ratio of deaths was, say, 13/38 in the non-HCQ group in the hypertension category, when expanded to 146 cases is the ratio made to be 50/146? That would seem to me to be a reasonable thing to do since the intention is to compensate for a likely higher rate of death in this category, which would unfairly inflate the mortality rate for the HCQ group since so many of them had hypertension.
Additionally to that I think you should also report the death ratios for each of the categories for which they are known to be high risk factors, such as hypertension. That is, for hypertension and other risk factors compare the death ratios for the HCQ and non-HCQ groups. There may be significant advantages or disadvantages that you may have missed if you only report the overall death ratios.
II.) The conclusion you drew from your study was that HCQ offered no benefit. But I found a key category in your study where the data did not support that conclusion, the patients on ventilators. COVID-19 patients on ventilators in general have poor outcome. In New York only 20% survive it:
https://www.usatoday.com/story/news/health/2020/04/08/coronavirus-cases-ventilators-covid-19/2950167001/
So it would be extremely important to find drugs that would improve that number.
Adding “deaths” to “intubated” complicates the analysis. So I looked just at intubated.
From your Table S1 in the Supplemental Appendix file (attached), we get:
For HCQ, 154 - 49 = 105 out of 154 survive, 68.2%
For non-HCQ, 26 - 17 = 9 out of 26 survive, 34.6%
So HCQ is twice as good for survival for intubated patients.
The 34.6% non-HCQ number in your study might be higher than the 20% New York number because the sicker patients were on HCQ in your study, leaving the comparatively healthier patients in your count for non-HCQ intubated patients. Likewise this would also deflate the the survival numbers for the HCQ intubated patients.
So focusing just on this category of intubated patients gives another example where you would want to break the cases up into the various risk factors and compare survival between HCQ and non HCQ groups. Then you could also calculated a more accurate death rate by using the method I described in part I.) above. Getting survival to ca. 70% using HCQ or perhaps higher would be extremely important.
III.) I was also extremely interested to see that all patients, HCQ and non-HCQ, took interferon. Interferon is a potent antiviral and anti cancer medication. There have been multiple studies now showing interferon shows benefit in treating COVID-19. Then the closeness of HCQ and non-HCQ numbers may have been because the interferon improved the outcomes of both groups. In order to assess the usefulness of HCQ, clearly that would be better illuminated if it were compared to a non-medicated control group.
No doubt for medical ethical reasons though, you didn’t want to leave any patients unmedicated, even the controls. Also, I presume you also wanted to give the HCQ group the interferon since the HCQ was an unproven medication. By the way Dr. Raoult has been criticized in his studies because he didn’t use controls. He says for medical ethics reason he didn’t want to leave patients unmedicated. I think you can see this is the same reason in yours and other studies assessing HCQ they always gave the control groups other medications.
But this leaves doubt about the conclusion that HCQ offered no benefit. A more accurate description would be the HCQ offered no ADDITIONAL benefit over using just interferon alone. Let me explain this another way. Suppose other medical professionals took literally the conclusion that HCQ offered no benefit. Then they might conclude they didn’t have to give these patients even under severe disease ANY medication. By making clear instead this was in comparison to interferon they would understand there might be other medications such as interferon that could work just as well or better.
Thank You,
Robert Clark
___________________________
Robert Clark
Dept. of Mathematics
Widener University
One University Place
Chester, PA 19013 USA
___________________________
I.) Propensity Score Matching.
In reference to your Table 1 (attached), how do the “propensities” go into the calculation of the likelihood of death? For hypertension, when expanding the non-HCQ no. from 38 to 146 for propensity, do you keep the same ratio of deaths to the total in the category?
For example, if originally, the ratio of deaths was, say, 13/38 in the non-HCQ group in the hypertension category, when expanded to 146 cases is the ratio made to be 50/146? That would seem to me to be a reasonable thing to do since the intention is to compensate for a likely higher rate of death in this category, which would unfairly inflate the mortality rate for the HCQ group since so many of them had hypertension.
Additionally to that I think you should also report the death ratios for each of the categories for which they are known to be high risk factors, such as hypertension. That is, for hypertension and other risk factors compare the death ratios for the HCQ and non-HCQ groups. There may be significant advantages or disadvantages that you may have missed if you only report the overall death ratios.
II.) The conclusion you drew from your study was that HCQ offered no benefit. But I found a key category in your study where the data did not support that conclusion, the patients on ventilators. COVID-19 patients on ventilators in general have poor outcome. In New York only 20% survive it:
https://www.usatoday.com/story/news/health/2020/04/08/coronavirus-cases-ventilators-covid-19/2950167001/
|
Despite the rush to secure more ventilators amid the coronavirus crisis, the fact is that they won't fix the problem. But they do buy patients time.
www.usatoday.com
|
So it would be extremely important to find drugs that would improve that number.
Adding “deaths” to “intubated” complicates the analysis. So I looked just at intubated.
From your Table S1 in the Supplemental Appendix file (attached), we get:
For HCQ, 154 - 49 = 105 out of 154 survive, 68.2%
For non-HCQ, 26 - 17 = 9 out of 26 survive, 34.6%
So HCQ is twice as good for survival for intubated patients.
The 34.6% non-HCQ number in your study might be higher than the 20% New York number because the sicker patients were on HCQ in your study, leaving the comparatively healthier patients in your count for non-HCQ intubated patients. Likewise this would also deflate the the survival numbers for the HCQ intubated patients.
So focusing just on this category of intubated patients gives another example where you would want to break the cases up into the various risk factors and compare survival between HCQ and non HCQ groups. Then you could also calculated a more accurate death rate by using the method I described in part I.) above. Getting survival to ca. 70% using HCQ or perhaps higher would be extremely important.
III.) I was also extremely interested to see that all patients, HCQ and non-HCQ, took interferon. Interferon is a potent antiviral and anti cancer medication. There have been multiple studies now showing interferon shows benefit in treating COVID-19. Then the closeness of HCQ and non-HCQ numbers may have been because the interferon improved the outcomes of both groups. In order to assess the usefulness of HCQ, clearly that would be better illuminated if it were compared to a non-medicated control group.
No doubt for medical ethical reasons though, you didn’t want to leave any patients unmedicated, even the controls. Also, I presume you also wanted to give the HCQ group the interferon since the HCQ was an unproven medication. By the way Dr. Raoult has been criticized in his studies because he didn’t use controls. He says for medical ethics reason he didn’t want to leave patients unmedicated. I think you can see this is the same reason in yours and other studies assessing HCQ they always gave the control groups other medications.
But this leaves doubt about the conclusion that HCQ offered no benefit. A more accurate description would be the HCQ offered no ADDITIONAL benefit over using just interferon alone. Let me explain this another way. Suppose other medical professionals took literally the conclusion that HCQ offered no benefit. Then they might conclude they didn’t have to give these patients even under severe disease ANY medication. By making clear instead this was in comparison to interferon they would understand there might be other medications such as interferon that could work just as well or better.
Thank You,
Robert Clark
___________________________
Robert Clark
Dept. of Mathematics
Widener University
One University Place
Chester, PA 19013 USA
___________________________
1 comment:
According to the "News in Brief" section of the AAAS's "Science" magazine (a respected reporting journal), the FDA has determined that hydroxychloroquine and chloroquine are ineffective at treating or preventing covid-19, while still a risk for some severe side effects. Aproval to distribute FDA stocks of these drugs for treating covid-19 has been withdrawn, while approval to run test studies is still in place.
Meanwhile, UK researchers have found an inexpensive and safe-to-use steroid (dexamethasone) that improves survival significantly for patients on ventilators or even just on oxygen. This was a big trial (a few thousand), and the results are very significant statistically. These results will soon be published.
From other sources, it seems that seriously-ill covid-19 patients show damage from serious inflammation, in the lungs and a variety of other organs and tissues. Thus, it makes sense that a steroid with anti-inflammatory properties should help. And according to those UK researchers, it does.
So the work to find treatments continues, pretty much just as it should. -- GW
Post a Comment