Ex-gay Study Rigged to Achieve Desired Results?
Mel Seesholtz assesses the weaknesses of the recent ex-gay study by Stanton Jones of Wheaton College and Mark Yarhouse of Regent University.
The Thomas Project: A study designed for specific results
A well-known, peer-reviewed research study of ex-gays that was conducted by Robert Spitzer in 2001 made fundamental mistakes of methodology, according to those peer reviewers: telephone interviews, no physical monitoring, slanted definitions of “success” and “change,” biased pool of subjects — in a single round of 45-minute telephone interviews. Was it really worth Jones and Yarhouse’s effort, and Exodus’ money, to spend five years repeating the same mistakes — this time conveniently skipping the peer review?
It is puzzling that Exodus and Focus on the Family claim to be pleased by study findings that — at best — only 11 percent of ex-gay program participants report a modest degree of bisexuality after 5 years. Who do they believe they’re fooling with talk of “change” in sexual orientation when — according to Christianity Today — nearly all the “successes” are bisexuals* who acknowledge they are constantly fighting off same-sex urges?
Seesholtz sums up the selectivity of the study, which appears to have utilized ex-gay activists ex-gays recruited by Exodus as subjects of the study:
A weak methodology that included only participants likely to yield the desired results: junk science encouraging potentially harmful “therapeutic” practices.
The study failed to provide objective scientific evidence of true change in sexual orientation. Seesholtz concludes:
That being said, should individuals have the right to seek “ex-gay” therapy? Yes: caveat emptor. Before entering into such programs, however, those offering such “treatments” should be required — ethically and legally — to tell prospective clients the scientific, medical facts.
I still do not understand why this study was released in book form rather than submitted to a major psychological research journal where it could be peer reviewed by psychologists. I am guessing that the major psych publications would find it too weak to publish.
The other thing that I do not understand is why it has to be so long. Surely this study could have been condensed to the size of a normal journal article with some proper editing.
For those who bought the book, how much did it cost? I wonder how much the authors are making on the sale of these books.
I’ve been very busy with work, but it appears that since I last caught up quite the conversation has been started on this thread!
First, to respond to your comments, Warren, I think it’s misleading to present this half-truth of the initial recruitments by Shidlo and Schroeder, though I’ve noticed that you have done this before on your own site. Due, in part, to the paucity of supportive data and the fact that conversion-oriented therapies are often framed in the context of empirically disproven ideas about sexuality and gender, Shidlo and Schroeder understandably began their study by assuming these were unhelpful therapies. When it became clear that some individuals felt helped by this, regardless of still being gay, they broadened their study. They began recruiting in ex-gay media sources for individuals who reported benefit, regardless of outcome, as well as a variety of “neutral” ads.
Additionally, I think it is important to also frame this research within the larger context of debates in the field regarding efficacy vs. effectiveness research. For you non-psychologists, efficacy research is intended to demonstrate that a therapy actually leads to the predicted changes in behavior, symptom reduction, etc. The “gold” standard of efficacy studies would involve randomly assigning participants into a control or a treatment group and demonstrating change. Recently, there has been a shift toward the use of an “intent-to-treat” analysis as opposed to a “survivor” analysis – in other words, follow everyone assigned to each group, not just the ones that follow a therapy through to the end!
Effectiveness research, on the other hand, looks at the real world application of those therapies. Something that works in a treatment study with depressed college sophomores may not work in the messy real world where depressions are tied up with a variety of other psychological and environmental factors. The Shidlo & Schroeder study explicitly invoked this debate through their title, as this debate began with a study of general therapy satisfaction published in Consumer Reports, with an accompanying article in American Psychologist (Seligman, 1995).
So it’s true to say that Shidlo and Schroeder say nothing about efficacy – in a well-controlled, randomized trial, who knows if therapy changes sexuality? It has never been tested, though the ethics of such a study would be seriously questioned. In terms of the study of effectiveness, Shidlo and Schroeder attempted to recruit a broad sample of all persons that had gone through such sexual reorientation programs, whereas the current and past Yarhouse and Exodus studies fail to even design a solid intent-to-treat protocol.
All that being said, I must add in relation to another comment that the idea it does not matter that some participants in this study had been enrolled in treatment for years prior is patently bizarre. Warren stated:
Of course they could not have failed just as easily – this statement represents a profound lack of experience with research. These participants had 2-3x the intervention, and had continued in a treatment that many participants discontinue. There is no reason to believe that this subsample is even representative of those who pursue change-oriented therapy.
Warren continued:
We’ll have to disagree here, Warren. As I’ve already clarified, to hold that this highly skewed sample involving a survivor analysis and inadequate justification for efficacy without controls, Jones and Yarhouse is clearly not in league with Shidlo and Schroeder. Warren is wrong to maintain otherwise.
Finally, encouraging efforts to change a trait (that the vast majority of scientists consider immutable and possibly inborn) flies in the face of Acceptance and Commitment Therapy. Yoking this pseudoscience to the bumper of a moving train will not help pull it along, Warren.