In this post I shall examine Milgrom’s opening and his comments on the evidence for homeopathy. I will be examining his arguments around: the scientific nature of homeopathy, its risks, the role of the profit motive and the influence of philosophy, in subsequent posts.
Sitting comfortably?
The summary starts with a familiar defence: “homeopathy has been in successful and continuous use for well over 200 years”. This makes the usual mistake of conflating two different arguments: efficacy and popularity. It is a common mistake to assume that the two go hand in hand. History tells a different story.
For example, medical bloodletting was both popular and ineffective for almost two-thousand years. In fact, it was positively harmful.
Milgrom starts to identify what he sees as the main accusations levelled against homeopathy, that it’s ‘unproven’, ‘unscientific’, and even ‘deadly’. These seem to be a fair representation of the problems with this delusion. The context within which he sees these attacks being made is a bit odd, “the globalised pharmaceutical industry which is itself in crisis, and a succession of UK governments seemingly supine in the face of legislation originating from the European Union.”
Of course, the pharmaceutical industry is a major bug bear of alternative ‘medicine’. I was surprised to see the UK government and the EU attract the ire of a homeopath.
Homeopathy in crisis?
Milgrom starts the essay proper with a lament. National Health Service (NHS) spending on homeopathic prescriptions fell by nearly 50% between 2005 and 2007. One of the five homeopathic hospitals funded by the UK taxpayer has been earmarked for closure. The, “flagship Royal London Homeopathic Hospital required an Early Day Motion and a debate in the House of Commons to temporarily guarantee its continued existence.”
There is a tone of entitlement in this piece: homeopathy has been available free on the NHS since its inception, so, by implication it should always be.
Now comes some confused whinging. Accusations that, “homeopathic remedies are ‘deadly’, yet no better than sugar pills” are “confusing”. It is surprising that Milgrom is so easily confused. As the offending articles he cites make clear, the reason is simple: taking entirely ineffective sugar pills for a serious illness can be dangerous or, even lethal. It’s really very simple: dangerous illness need real medicine.
In making this contention Milgrom fails either to understand, or fairly represent, the views of the authors he cites. For instance, he references an article by Nick Cohen. This is very clear as to why sugar pills can be lethal; the comments were made in the context of claims by named homeopaths to be able to treat AIDS. Here are the opening two paragraphs:
“On 1 December, faith healers will meet at Roots & Shoots in south London to discuss how to treat Aids with magic pills. They won’t call themselves faith healers, of course, or shamans or juju men. They will present themselves as ‘homeopaths’: serious men and women whose remedies are as good as conventional medicine.
According to the advance publicity, Hilary Fairclough, a homeopath endorsed by no less than Jeanette Winterson, will describe the ‘impressive’ results from her clinic in Botswana. Harry van der Zee, co-founder of the Amma Resonance Healing Foundation, will say that ‘in just a few days or weeks’ African Aids patients he treated became ’symptom-free and able to return to their jobs and schools or to look after their children again’. All in all, the Society of Homeopaths promises to provide ‘fascinating insights’ for World Aids Day.”
I would say that’s very easy to understand and not at all confusing. Next Goldacre (2007) is cited. This very balanced article is similarly clear as to where the problems lie (references omitted):
“There are also more concrete harms. A routine feature of homoeopaths’ marketing practices is to denigrate mainstream medicine. One study found that half of all homoeopaths who were approached advised patients against the measles, mumps, and rubella vaccine for their children. A television news investigation found that almost all homoeopaths who were approached recommended ineffective homoeopathic prophylaxis for malaria, undermined medical prophylaxis, and did not even give simple advice on bite prevention. Undermining medicine is a wise commercial decision for homoeopaths, because survey data show that a disappointing experience with mainstream medicine is one of the few features to regularly correlate with a decision to use alternative therapies. But it might not be a responsible choice.
Homoeopaths can undermine public-health campaigns; leave their patients exposed to fatal diseases; and, in the extreme, miss or disregard fatal diagnoses. There have also been cases of patients who died after medically trained homoeopaths advised them to stop medical treatments for serious medical conditions.”
Again, it’s hard to see what is confusing about Goldacre’s position. Milgrom also manages to overlook Goldacare’s discussion as to how homeopathy might be clinically useful – he’s not such a nasty sceptic after all.
Milgrom’s final reference in support of his contention that critics of homeopathy are effectively trying to ‘have their cake and eat it’ is an article by Samarasekera (2007). As expected, the source of Milgrom’s confusion is elusive. The piece quotes Michael Baum, making a very clear point, “People say homoeopathy cannot do any harm but when it is being promoted for HIV then there is a serious problem”. And, after mentioning the Sense About Science exposé of homeopathic clinics and pharmacies who were willing to sell homeopathic pills to protect against malaria, David Colquhoun is quoted being equally clear, “Making false claims about treating colds is one thing but it is quite another thing to make false claims about malaria”.
One of Goldacre’s other contentions is that homeopaths have a tendency to cherry-pick and misrepresent evidence (Goldacre, 2007). Unfortunately Milgrom falls into these traps when he starts discussing the evidence for homeopathy.
Un, deux, trois, nous irons au bois. Quatre, cinq, six, ceuillir des cerises
Milgrom’s first line of attack to refute the charges against homeopathy is to argue that, “Apart from several hundred years of clinical case histories, there are many good quality scientific trials and meta-analyses showing that homeopathy can demonstrate clinically observable effects over and above placebo”.
This is not a promising start. First, case histories are low-grade evidence. Subject to the vagaries of observer bias, expectation effects and the natural history of a disease – case histories are starting points not destinations. Second it’s suspicious that instead of citing these “good quality […] trials and meta-analyses” Milgrom cites two cherry-picking reviews put together by advocacy organisations.
European Network of Homeopathy Researchers: An overview of positive homeopathy research and surveys, March 2007. www.homeopathy-soh.org/whats-new/documents/POSITI.PDF
Alliance for Natural Health: Homeopathy. Modality: Homeopathy. www.anhcampaign.org/practitoners/homeopathy.
What do the meta-analyses really say? Homeopaths often refer to Kleijnen, Knipschild and Ter Riet (1991) but tend to accentuate the positive and ignore the caveats. They concluded:
“At the moment the evidence of clinical trials is positive but not sufficient to draw definitive conclusions because most trials are of low methodological quality and because of the unknown role of publication bias. This indicates that there is a legitimate case for further evaluation of homoeopathy, but only by means of well performed trials.”
This is in no way a demonstration that homeopathy is effective. Linde et al (1997) is another favourite of homeopaths, who often cite the first sentence of the conclusion but omit the second.
“The results of our meta-analysis are not compatible with the hypothesis that the clinical effects of homoeopathy are completely due to placebo. However, we found insufficient evidence from these studies that homoeopathy is clearly efficacious for any single clinical condition. Further research on homoeopathy is warranted provided it is rigorous and systematic.”
As Milgrom is contending that the evidence supports the clinical use of homeopathy, the authors’ caution against drawing such a conclusion is important:
“Our study has no major implications for clinical practice because we found little evidence of effectiveness of any single homoeopathic approach on any single clinical condition.”
Re-analyses of the data demonstrated that this work had been overconfident about its ability to account for publication bias and assess the methodological quality of included studies. Subsequently, many of the same authors revisited their data (Linde et al, 1999) and concluded that:
“[…] in the study set investigated, there was clear evidence that studies with better methodological quality tended to yield less positive results. Because summarizing disparate study features into a single score is problematic, meta-regression methods simultaneously investigating the influence of single study features seem the best method for investigating the impact of study quality on outcome.”
Finally, Cucherat et al (2000) also showed that the evidence just does not support using homeopathy, as opposed to researching it:
“[…] sensitivity analysis showed that the P value tended towards a non-significant value (P = 0.08) as trials were excluded in a stepwise manner based on their level of quality. […]
Conclusions: There is some evidence that homeopathic treatments are more effective than placebo; however, the strength of this evidence is low because of the low methodological quality of the trials. Studies of high methodological quality were more likely to be negative than the lower quality studies. Further high quality studies are needed to confirm these results”
So, Milgrom spectacularly fails to refute the central charge levelled at homeopathy. His use of reviews that are cherry-picked, written by advocacy groups and exist outside of the peer-reviewed medical literature is deeply flawed. The major meta-analyses of the literature provide no justification for using homeopathy in clinical practise.
The cherry-picking continues as the essay ventures into the area of what passes for basic homeopathic science. He cites the discredited Nature paper on basophil degranulation. (Davenas et al., 1988) and an alleged replication* by Belon et al (2004).
In typical cherry-picking style he omits a failed attempt at replication published in Nature during 1993 (Hirst et al., 1993). He also neglects to mention another failed replication by Guggisberg et al (2005). This concluded:
“We were not able to confirm the previously reported large effects of homeopathic histamine dilutions on basophil function of the examined donor. Seemingly, minor variables of the experimental set up can lead to significant differences of the results if not properly controlled.”
This seems to get to the heart of the matter: poor experimental control. Comments made by one of the authors of the debunked Nature paper (Beauvais, 2008) strengthen this impression:
“[...]The main issue was that in some circumstances, “effect” and “no effect” were randomly distributed regardless their origin (negative or positive samples) [...] the results of blinded samples were almost always at random and did not fit the expected results: some “controls” were active and some “active” samples were without effect on the biological system [...]“
In other words once the experimenters were not aware what the ‘expected’ result was, the results were random. All the experimenters were measuring was bias; their own.
“This has been shown to be thoroughly biased [14–17], a view reinforced by two recent studies further demonstrating the Lancet meta-analysis as seriously flawed [18, 19]. In addition, this meta-analysis broke the Lancet’s own stringent guidelines on methodological and publication transparency [20], leading one to question why it ever appeared in such an eminent journal.”
Milgrom’s first set of references [14 – 17] all come from the CAM literature. As the appearance of this deeply flawed essay in a peer reviewed CAM journal shows, they don’t seem to be particularly careful in what they publish: so I’ll not waste my time on them. The truth of the matter is that various critics had letters published in the Lancet (Fisher et al.,2006; Linde and Jonas, 2006; Walach et al., 2006; Dantas, 2006). The authors’ reply (Shang et al, 2006) addressed the concerns raised. No substantial criticism remained unaddressed**.
Homeopaths continue to make all sorts of criticisms of this work – popular myths include that it ignored key papers or that the authors never disclosed which trials made it into their final sub-group. These are generally the result of a failure to understand*** (or sometimes even read) the paper and the authors’ reply. (For further details see various posts‡ on this, or Paul Wilson’s blog)
The two recent studies that Milgrom cites do not deliver what he claims. Lüdtke and Rutten (2008) actually came to a rather mild conclusion, “Because of the high heterogeneity between the trials, Shang’s results and conclusions are less definite than had been presented.” This is hardly evidence of deep flaws. What they actually found was that if you make post-hoc choices about inclusion criteria and analysis methods you can get different results. This is not surprising and is why research studies should have their analysis methods set beforehand – otherwise researchers might unconsciously influence the results of their studies by selecting methods which give them the result they want.
A careful analysis by David Gorski shows that this paper actually confirms one of Shang’s key findings: evidence of bias was found in the sub-set of higher quality studies that was missing from their final set of eight most reliable trials. The point of Shang’s work was to arrive at a set of least biased trials on which to make judgements of efficacy. He concluded:
“[…] whenever one investigator “reanalyzes” the dataset of another investigator, they virtually always have an axe to grind. That doesn’t mean it isn’t worthwhile for them to do such reanalyses or that they won’t find serious deficiencies from time to time, but you should always remember that the investigators doing the reanalysis wouldn’t bother to do it if they didn’t disagree with the conclusions and weren’t looking for chinks in the armor to blast open so that they can prove the study’s conclusions wrong. In this, Lüdtke and Rutten failed.”
Milgrom’s second paper (Rutten and Stolper, 2008) is actually a more homeopath-friendly - therefore less objective – report of the same reanalysis: a bit of double-counting. The excellent Paul Wilson critiqued this on his blog (here and here); this was subsequently published (Wilson, 2009). The authors’ reply (Rutten and Stolper, 2009) signally failed to engage with his substantial criticisms.
Shang et al (2005) has been subjected to a barrage of criticism. Some of it is fair; most is partisan and ill-informed. A re-analysis by homeopaths failed to show any errors. The most that can be said is if you do it differently you can get different results. This is an entirely trivial conclusion.
As for Milgrom’s allegations of lack of appropriate transparency, he had a point. The original paper did not fully disclose the identities of the trials analysed. This should have been included in the original paper. However, the error was put right in December 2005. One thing that Lüdtke and Rutten (2008) showed is that it is possible to go back, reconstruct Shang’s analysis and get the same results. Any lack of transparency, regrettable as it is, does not change what they found:
“Biases are present in placebo-controlled trials of both homoeopathy and conventional medicine. When account was taken for these biases in the analysis, there was weak evidence for a specific effect of homoeopathic remedies, but strong evidence for specific effects of conventional interventions. This finding is compatible with the notion that the clinical effects of homoeopathy are placebo effects.”
Guilty as charged
Critics of homeopathy rightly point to lack of credible evidence that it has specific clinical effects. As we have seen, this is consistent with the evidence provided by major meta-analyses. It is also clear that the basic research offered up by homeopaths has been shown to be unreliable.
Some sceptics have also made it very clear that homeopaths advocating the use of ineffective homeopathic treatments for serious illnesses (like AIDS and malaria) poses risks. They have been precise about both the illnesses and homeopaths involved: the risk is not in the pills, but in their uselessness; and the attitudes to medicine promoted by some homeopaths (like discouraging vaccination).
All Milgrom offers in this essay is a spurious smokescreen of “confusion”, lists of studies cherry-picked by advocacy organisations and an insubstantial critique of Shang et al (2005).
It is baffling that a man of education and intelligence could think that this was any kind of serious refutation of the charges. Equally, it is revealing that such an error-strewn analysis could make it into a peer-reviewed journal. Then again, it’s a CAM journal.
Next, I’ll look at Milgrom’s attempted refutation of “The claim that homeopathy is deadly”.
Also in this series
Watch this space.
Disclaimer
I am not a doctor. This does not constitute medical advice. If you need that consult a properly qualified and registered medical practitioner.
These are my opinions, but I try to make sure that what I write is both accurate and fair. If you think that I have got anything wrong please let me know. If you are right I will happily change what I have written.
Notes
*See The Great DBH Rant blog for a devastating critique of this work.
**Contrary to popular homoeomythology the authors subsequently made available a complete list of all the trials analysed in the study, including which made it into various sub-groups.
*** I have had a go at explaining what the authors actually did here. If you are a homeopath, or apologist for homeopathy, please read this before offering any criticism.
‡In Dangerous delusions Jayney Goddard repeats the myth that “that they didn’t disclose the identities of the final eight studies.” In Making your own reality I tried to get an article corrected in which Dana Ullman claimed that the study “did not include any of David Reilly’s research”. In part two of this post I dealt with Ullman’s further criticisms of Shang. In Spying on Shang I took apart a rather silly criticism by a homeopath named Clive Stuart. Shang’s secret – the hydra of homoeomythology finds Milgrom pretending in 2008 that the authors had not disclosed the identity of the studies they analysed, although they did this in December 2005! Homeopathy and the Absence of Evidence deals (among other things) with Dr Damien Downing’s misconceptions. Finally, The Myth of The Secret Eight, was my first foray into the debate over this meta-analysis.
References
Beauvais F. Memory of water and blinding. Homeopathy. 2008 January;97(1):41–42. Available from: http://dx.doi.org/10.1016/j.homp.2007.10.001.
Belon P, Cumps J, Ennis M, Mannaioni PF, Roberfroid M, Sainte-Laudy J, et al. Histamine dilutions modulate basophil activation. Inflamm Res. 2004 May;53(5):181–188. Available from: http://dx.doi.org/10.1007/s00011-003-1242-0.
Cucherat M, Haugh MC, Gooch M, Boissel JP. Evidence of clinical efficacy of homeopathy. A meta-analysis of clinical trials. HMRAG. Homeopathic Medicines Research Advisory Group. European journal of clinical pharmacology. 2000 April;56(1):27–33. Available from: http://view.ncbi.nlm.nih.gov/pubmed/10853874.
Dantas F. Are the clinical effects of homoeopathy placebo effects? The Lancet. 2006 January;366(9503):2083. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67880-4.
Davenas E, Beauvais F, Amara J, Oberbaum M, Robinzon B, Miadonna A, et al. Human basophil degranulation triggered by very dilute antiserum against IgE. Nature. 1988 June;333(6176):816–818. Available from: http://dx.doi.org/10.1038/333816a0.
Fisher P, Berman B, Davidson J, Reilly D, Thompson T. Are the clinical effects of homoeopathy placebo effects? The Lancet. 2006 January;366(9503):2082–2083. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67879-8.
Goldacre B. Benefits and risks of homoeopathy. The Lancet. 2007 November;370(9600):1672–1673. Available from: http://dx.doi.org/10.1016/S0140-6736(07)61706-1.
Guggisberg AG, Baumgartner SM, Tschopp CT, Heusser P. Replication study concerning the effects of homeopathic dilutions of histamine on human basophil degranulation in vitro. Complementary Therapies in Medicine. 2005 June;13(2):91–100. Available from: http://dx.doi.org/10.1016/j.ctim.2005.04.003.
Hirst SJ, Hayes NA, Burridge J, Pearce FL, Foreman JC. Human basophil degranulation is not triggered by very dilute antiserum against human IgE. Nature. 1993 December;366(6455):525–527. Available from: http://dx.doi.org/10.1038/366525a0.
Kleijnen J, Knipschild P, Ter Riet G. Clinical trials of homoeopathy. BMJ (Clinical research ed). 1991 February;302(6772):316–323. Available from: http://view.ncbi.nlm.nih.gov/pubmed/1825800.
Linde K, Jonas W. Are the clinical effects of homoeopathy placebo effects? The Lancet. 2006 January;366(9503):2081–2082. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67878-6.
Linde K, Scholz M, Ramirez G, Clausius N, Melchart D, Jonas WB. Impact of study quality on outcome in placebo-controlled trials of homeopathy. Journal of clinical epidemiology. 1999 July;52(7):631–636. Available from: http://view.ncbi.nlm.nih.gov/pubmed/10391656.
Linde K, Clausius N, Ramirez G, Melchart D, Eitel F, Hedges LV, et al. Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials. Lancet. 1997 September;350(9081):834–843. Available from: http://view.ncbi.nlm.nih.gov/pubmed/9310601.
Lüdtke R, Rutten AL. The conclusions on the effectiveness of homeopathy highly depend on the set of analyzed trials. Journal of Clinical Epidemiology. 2008 December;61(12):1197–1204. Available from: http://dx.doi.org/10.1016/j.jclinepi.2008.06.015.
Milgrom LR. Under Pressure: Homeopathy UK and Its Detractors. Forsch Komplementmed. 2009 September;16(4):256–261. Available from: http://content.karger.com/ProdukteDB/produkte.asp?Aktion=ShowAbstract&ArtikelNr=228916&Ausgabe=248719&ProduktNr=224242
Rutten ALB, Stolper CF. The 2005 meta-analysis of homeopathy: the importance of post-publication data. Homeopathy : the journal of the Faculty of Homeopathy. 2008 October;97(4):169–177. Available from: http://dx.doi.org/10.1016/j.homp.2008.09.008.
Rutten ALB, Stolper CF. Reply to Wilson. Homeopathy. 2009 April;98(2):129. Available from: http://dx.doi.org/10.1016/j.homp.2009.01.002.
Samarasekera U. Pressure grows against homoeopathy in the UK. The Lancet. 2007 November;370(9600):1677–1678. Available from: http://dx.doi.org/10.1016/S0140-6736(07)61708-5.
Shang A, Huwiler-Müntener K, Nartey L, Jüni P, Dörig S, Sterne JA, et al. Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet. 2005;366(9487):726–732. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67177-2.
Shang A, Jüni P, Sterne JAC, Huwiler-Müntener K, Egger M. Are the clinical effects of homoeopathy placebo effects? Authors’ reply. The Lancet. 2006 January;366(9503):2083–2085. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67881-6.
Walach H, Jonas W, Lewith G. Are the clinical effects of homoeopathy placebo effects? The Lancet. 2006 January;366(9503):2081. Available from: http://dx.doi.org/10.1016/S0140-6736(05)67877-4.
Wilson P. Analysis of a re-analysis of a meta-analysis: in defence of Shang et al. Homeopathy : the journal of the Faculty of Homeopathy. 2009 April;98(2). Available from: http://dx.doi.org/10.1016/j.homp.2008.12.003.
Acknowledgements
dvnutrix for pointing this nonsense out to me, and the writer of The Great DBH Rant blog for their insights into Belon et al (2004).
Nessun commento:
Posta un commento