Tuesday, November 10, 2015

The Business of Psychological Questionnaires

Questionnaires are certainly in vogue in mental health research.  Often they are referred to in technical-sounding jargon, for example it is common to call a questionnaire an "instrument"  or a "measurement tool."

There are good reasons to have well-standardized questionnaires.  In research, it is useful if people across the world are all using a similar type of questionnaire, so that comparisons can be made more easily and clearly.

In psychotherapy or other mental health practice, there is evidence that obtaining regular feedback from patients or clients can be valuable to improve the quality of the therapy, and to prevent mistakes.  One of the leaders in showing the importance of this is Michael Lambert, an esteemed psychologist and psychotherapy researcher from Brigham Young University.  In a nutshell, his research shows us that problems can occur in psychotherapy without the therapist realizing it:  the patient or client could be developing new symptoms, detaching or losing interest in the therapy, feeling upset or disappointed with the therapist, or even developing a life-threatening emergency, but the therapist may not know this, because it is not talked about or asked about in the session.  This could be because the patient is inhibited to share this information, but it could also be simply because the problem was never inquired about.  In therapy sessions, just like with any other interaction, one can follow a certain narrative pathway habitually, therefore missing things that could be quietly going wrong in the background.

So Lambert has developed a questionnaire called the OQ-45, which consists of 45 simple questions covering everything from mood, anxiety, relationship satisfaction, loneliness, drinking, family life, work life, cognition, and physical health.  The idea is for patients or clients to fill in this questionnaire frequently, maybe even before every therapy appointment, so that no potential evolving problem area would be "missed."   The questionnaire would only take a few minutes to fill out, and could be done in the waiting room before an appointment.    Samples of the OQ-45 can be found in an internet search.  

I believe that this type of questionnaire is useful.  Certainly we have to respect Lambert's many years of research, to acknowledge that feedback of this type can improve therapy.

But the therapeutic benefit of this is not due to some special property of the questionnaire itself!  And the therapeutic benefit does not require the sophisticated statistical analysis that is offered to purchasers of the questionnaire!  The benefit of this is simply to do a review of symptoms regularly with patients or clients.  

Questionnaires in psychology have become a business.  For hundreds of dollars, one can sign up to receive copies of a questionnaire, scoring manuals, or perhaps an on-line entry and scoring package, which may produce attractive graphs of results.

I believe that it is absurd--in most cases--to have to pay for something like this.  The therapeutic principle here is of simply keeping track of a wide range of symptoms or problems systematically.   The technology here is not a sophisticated x-ray machine or microscope -- rather, they are sets of simple questions such as "I'm a good person" or "My body hurts" (to be rated from 0-4).

I have jokingly thought of creating a questionnaire, to be marketed, with a full statistical analysis package and online access, called the "How Are You Doing" instrument (the HAY-D-1).  It would consist of a single question, "How are you doing?"  with the opportunity to choose from one of 5 responses.    Perhaps there could be a published article demonstrating its reliability, validity, and correlations with other established research instruments. 

Understandably, many researchers have worked long and hard to show useful results from their work.  And it could be very desirable for them to have a way to earn a financial reward from the fruits of their labor.  I suppose, in a free society, it is quite reasonable for people to attempt to sell such things, if people are willing to buy them.

But when there is this type of marketing and financial dealing going on, it can increase biases on the part of both the seller and the buyer.  The buyer, having paid good money for questionnaires or "instruments," is more likely to think highly of their acquisition, due to cognitive bias (think again of Daniel Kahneman's work showing such effects).  Perhaps therapists are more likely to rely on such purchased questionnaires rather than simply creating their own.

I think it could be useful, if questionnaires are to be used at all, to create custom symptom review questions.  There is also some evidence that questions about the therapeutic alliance could be pertinent to therapeutic progress; these are absent from many symptom review surveys, including the OQ-45.

A nice idea in CBT is to have the clients or patients be actively involved in assessing and planning their own progress, instead of having the therapist be the "assessor."  So, it could be a useful therapeutic exercise for clients or patients to design their own questionnaires, using their own language, and their own scale!  The therapist could encourage and suggest a wide range of categories of questions to be followed, covering areas of physical, social, occupational, cultural, and psychological health, as well as a category about the therapeutic alliance, but the questions themselves could be designed by the client or patient!    If statistical analysis was felt to be interesting or useful, we could easily design a simple app to create graphs, or use a spreadsheet -- we would not have to pay an extra fee for this!

So I support the idea of regularly conducting broad symptom reviews in psychotherapy, but I do not believe it is necessary to buy questionnaire packages.  It could be even better to design one's own package, or collaborate with a patient or client to design a custom, personalized survey.  


Anonymous said...

Fabulous blog.

You're onto something with HAY-D-1.

Have you read Kimberly Emmons' _Black Dogs, Blue Words: Depression in the Age of Self-Care_? Interesting work on (a) the psychological questionnaire embedded within the pharmaceutical advertisement, and (b) the "DIY" psychological questionnaire that is distinct from an advertisement, sometimes (but not always) funded by a pharma company, and which generally seems to culminate in the suggestion that one should probably talk to one's doctor. Emmons says something along the lines of "we are quizzing ourselves sick."

For Emmons, the phenomenon of depression begins long before the patient enters a doctor's office and gets a diagnosis of depression. But neither does "depression" the disease/illness begin, exactly, with the embodied experience that later gets described as depression. She makes the complex argument that depression is a "rhetorical illness," or an illness of a symbolic action.

To be clear, Emmons is not commenting on whether the thing/experience/event/phenomenon that later gets described as "depression" exists, nor is she commenting on whether this thing/experience/event/phenomenon is profoundly painful.

She is interested in theorizing *when* the thing/experience/event/phenomenon becomes "depression," of what essence the diagnosis is, and why there is so much of the diagnosis. Her theory is that the diagnostic process is initialized at home, via behaviors cultivated by questionnaires like (a) and (b) described above. The behaviours that these questionnaires cultivate could be described as anticipatory and interpretive: people (and especially women) are taught to continually monitor how they are feeling, with a very strong sense of how they "ought" to be feeling. They are encouraged to watch for/recognize "symptoms of depression" in the self. It is at this moment-- when one's experience is processed/configured/rendered/interpreted (we're taught to say "recognized") as a possible "symptom"-- that the diagnosis of "depression" begins its life.

Discourse-- language, stories, symbols, media, etc-- does things. It has a material power. Emmons is interested in how the "self-doctoring" behaviours cultivated by the questionnaire are the *mechanism* by which the serotonin story, and other biomedical stories of depression, materialize, and get popularized. Depression-related discourse and genres = depression-constituting discourse and genres.

The thing that would make the OQ-45 especially different from the HAY-D-1, to work with your very generative joke, is that the OQ-45 presumes the existence of labels, whereas a HAY-D-1 questionnaire would be a human exchange that does not presume labels. (Of course, one would still, need to work to counter the label rhetoric implied by the clinical setting.)

GK said...

Thank you for your comment!

And for the reference to Emmons: I am excited to welcome insights from a scholar who has a focus of expertise in language and rhetoric. This speaks to another passionate belief of mine, that modern scholarship has such fragmentation that potential insights from one field may often not become part of a larger discourse affecting another field, simply because the specialists in both areas are intellectually (and often physically) distant from one another. Also, if there is a person being targeted by some kind of marketing scheme (with a goal, for example, of selling medication or a psychotherapy regime or a questionnaire), then the fragmented discourse could leave this person less empowered to make the best choice for self-care. We need more scholarly collaboration, and also a focus on countering the powerful marketing forces that we are all subject to (it is not only a particular demographic of "consumers" that are targeted by marketing efforts, it is physicians, therapists, and scholars who can be subject to these influences as well, often unconsciously).

But this observation could speak to a compassionate understanding of some of the problems in the current system -- that perhaps the mechanism by which unwelcome and economically-driven "therapeutic" strategies are harmful is rhetorical, rather than necessarily "biochemical." Just as with antibiotics, for example (which have saved the lives of millions), we must protect ourselves against a rhetorical belief that every infection must be treated with a course of a new antibiotic. So by focusing on communicative freedom, we can protect our health -- and most likely use far fewer antibiotics -- while still encouraging pharmaceutical research to find better treatments for resistant infections without fueling a public health catastrophe.

GK said...

In our discussion of quizzes and questionnaires, I am reminded of another situation where we see powerful biases and marketing at play: in the area of continuing medical education (CME) for practicing physicians.

It is necessary for physicians to demonstrate continuing education efforts (which is reasonable and important). But of course this is big business as well: expensive, lavish conferences, as well as journals, websites, etc. Well-known "specialists" conduct online lectures about various subjects.

When one encounters educational material of this sort, very often there is a quiz at the end. It is clear that doing quizzes is a valid educational strategy. If you are reading a text of any sort, it is optimal to quiz yourself about it afterwards if you want to enrich your learning.

But the trouble with these quizzes is that the material in the presentation may have been quite biased to begin with. If you answer the questions in the quiz correctly, it merely shows that you have internalized the content of the lecture accurately. This, in turn, would give the quiz-taker a feeling of satisfaction for a job well-done. Most physicians come from a background of academic achievement, in which a good deal of self-esteem has come from performing well on tests.

With these quizzes, you will not receive educational credit (a necessary requirement to maintain your professional license) unless you complete the quiz with an adequately high mark!

So doing quizzes following an educational presentation, and getting high marks, could be understood as a tool of biased persuasion. The quiz, in this case, risks being a manipulative rhetorical device rather than a facilitator of free thought.

As I write this, it occurs to me that this type of dynamic occurs too often in other spheres of education as well. Evaluation of students should, in my opinion, always have a view to foster intellectual freedom and a willingness to question.
Even in some subjects such as physics and mathematics, it could be more meaningful to encourage students to develop "rhetorical power" in their skills, rather than merely a technical ability to answer a certain type of problem correctly.

Anonymous said...

What a fabulous response! I am with you on the issues of fractioning scholarship.

You have organically arrived at Emmons' conclusion: she actually culminates her text in a proposal called "rhetorical self-care." What she has in mind is the fostering of a special kind of ability within the individual to be rhetorically literate-- to recognize what kinds of rhetoric (that is to say, discourses, motives, and/or symbolic actions) are at play in any given medical situation, working away at one. She thinks this kind of rhetorical literacy can help the individual combat coercive forces, and help the individual to make therapeutic choices that best attend to the individual's needs. Rhetorical self-care does no attach itself to any particular paradigm for depression. Ultimately, she calls it "good medicine." All of this can be found in her concluding chapter, "Toward a Rhetorical Care of the Self."

I personally find myself enacting rhetorical self-care in a lot of ways.
Of course, rhetorical self-care is predicated on the notion of individual choice and intellectual freedom.

...And these notions are not arhetorical.

I shall deviate from our interesting discussion on the questionnaire to address this other question of patient freedom that seems to structure your post and Emmons' text.
Here is what I mean.
You probably know about flibanserin's massive campaign to get the "female viagra" approved. The main message of the campaign was, "don't be sexist; give women the choice that men have." Well, you probably also know that there are real concerns about a) the safety of flibanserin; about b) the efficacy of flibanserin; about c) the legitimacy of the phenomenon we've begun to call "female sexual dysfunction"; and about d) the potentially serious harm (of a coercive quality) involved in telling a woman she should be having more sex than she wants to.

[My position on flibanserin as harmful and problematic is influenced by these sources: The New View Campaign (http://www.newviewcampaign.org/); this letter (https://www.nwhn.org/wp-content/uploads/2015/07/Senator-Mikulski-Letter-about-Flibanserin-June-2015-Copy-1.pdf); and the film Orgasms Inc. (http://orgasminc.org/).]
Ultimately, though there are so many issues with flibanserin, we (and the FDA) were persuaded by the compelling rhetoric of the assertion "it's my choice to take flibanserin."
But just because an individual has the choice to take flibanserin or not, doesn't mean that flibanserin is somehow now free from coercion, from misinformation... that it is free from harm.

So, I'm just hoping to emphasize the second part of your assertion here: it is meaningful to "foster intellectual freedom and a willingness to question." Perhaps what we need is to promote rhetorical self-care and intellectual freedom, while constantly drawing attention to the forces that limit our choices and freedoms.

I think this has relevance for psychiatry, too. Perhaps the progressive, critical psychiatrist wants to assert the individual's freedom to request, accept or decline a psychiatric label, or medication, but it is is important that we recognize the lack of choice available to people in such situations. Say a woman sees an ad that promises she can be a better mom and wife if she changes affect through an antidepressant medication. That woman who comes into your office may appear to choose to request a medication, but what choice does she have, really? Or a woman comes in and says she's 50 years old, isn't having enough sexual desire with her husband she's been married to for 25 years, but believes it is important that she be desired by and desire her husband more, sexually. So she chooses to request flibanserin. Where are the limits in her choice? How shall the psychiatrist who is dedicated to respecting individual choice to proceed?
Am I making sense here?

GK said...

I think there are frequent daily examples of similar dynamics going on with personal choices: people are influenced to consider acquiring various commercial items, engaging in various recreational pursuits (e.g. "parties"), listening to various types of music, wearing particular styles of clothing, ingesting various substances (e.g. beer, Coca-Cola, or--as a more benign example--water from plastic bottles), or even making various career choices, all in the context of what is often quite substantial peer pressure from friends, classmates, family members, or other members of society. The motive behind the pressure may itself simply be following waves of consumer trend, though it may be guided by sincere intentions of helpfulness. The question of a particular drug etc. is yet another example of a personal choice loaded with interpersonal and social pressure context.
But what to do with this? A respectable role for a psychiatrist should be, I think, to facilitate some sense of insight about the dynamics around choices of all types. However, I recognize that even this notion -- founded in the Freudian ideas around unconscious motivation -- carries biases of its own, which are obscured by an artificial inflation of the therapist's confidence of being "interpretive" or correct.
At the very least, I think and hope that all of us can spend some extra moments of contemplation, to reflect upon hidden influences upon our choices. Not all of these influences are necessarily "negative" or malevolent (though some clearly are), but I think it is important, in the process of living freely and healthily, to increase our awareness of how and why we choose.