Meta-analysis

Meta-analysis is a powerful technique for summarizing data across many research studies.  For example, to understand the role of psychotherapy or antidepressants to treat depression, a meta-analytic study could give us our best starting point to estimate the effect size.

But the meta-analytic method has a prominent weakness, what I would call dilution:

Suppose that one is doing a meta-analysis of the effectiveness of surgery vs. supportive care for treating abdominal pain.   Many studies might show that surgery is remarkably effective, yet others would show no difference, or even a negative effect, compared to supportive care.  The meta-analysis could average these out, and conclude that there was little difference.    The reason for the dilution is that there are some specific types of abdominal pain, with specific causes, which are best treated surgically (e.g. appendicitis).    Many other types of abdominal pain settle down on their own, or require simple supportive measures.  In the past, it was often difficult to determine whether a patient definitely had appendicitis or not, in the early stages of the illness.   Therefore there would have been many unnecessary appendectomies, and many other cases of ruptured appendicitis operated on too late.

Similarly, in psychiatry, I think it is probably true that there are particular subtypes of depression (or other diagnoses), which respond much better to psychotherapy, or much better to a particular medication, or which might settle down completely on their own with no help at all.   At present, our diagnostic schemes do not help us very much to differentiate between these groups.  We often assume that mild depressions are best treated with psychotherapy, and severe depressions are more likely to need medication treatments.  While there is evidence that supports this assumption, it is not invariably true:  some cases of mild depression persist for long periods of time, do not improve with psychotherapy, but may improve dramatically with a medication trial.  Conversely, some severe cases of depression may not respond well to medications, but improve dramatically with psychotherapy (sometimes a very particular type of psychotherapy).

An ongoing area of research must be to improve our ability to predict the optimal treatment strategy.  I suspect that in most cases, this strategy will involve some combination of psychotherapy, medication, and practical social support.    I think that the science to help us in this task is more likely to come from genetics, and less likely to come from more sophisticated questionnaires or symptom scales.

The search for these answers is confounded, in psychiatry, by a very high risk of placebo-like psychological effects, which must be addressed by studies which have very careful placebo controls and active placebo controls.


For example, many patients are understandably attracted by a very "high-tech" or "advanced science" approach to treating their illness.  So we have some clinics which offer sophisticated technology, such as neurofeedback, PET imaging, genomic analysis, etc.  While these technologies are interesting, and possibly very useful, they also carry a sort of "guru effect."  A PET scan yielding exciting images showing metabolic changes in the brain, accompanied by a detailed diagnostic report, could be much more persuasive than reading the exact same report without the images.    Therefore the PET imaging could act as a marketing tool, to cause the person to take the report more seriously, irrespective of whether the imaging actually shows something of true scientific relevance.  It would be like visiting a fortune-teller, but receiving actual images of your brain which are referred to in the fortune-teller's predictions about you.   It would be especially convincing!    Similarly, with neurofeedback, the dazzle of the technology could cause people to take the therapeutic tasks more seriously, causing improvement separate from the independent benefit of the technique.   I am particularly concerned about the risk of bias with these techniques, because some clinics or private practitioners are charging very high fees for patients to have them.  This is an environment in which selective glowing testimonial accounts could distort a reasonable summary of the data.

In order to conduct research properly with these new modalities, we must have very careful active placebo groups.  In a neurofeedback study, for example, there should be sham neurofeedback which generates a similar type of interactive therapeutic task, with a similar degree of technological dazzle.