Misplaced Pages

Revision as of 23:07, 23 September 2012

The Dodo bird verdict is a controversial phenomenon in psychology which states that all psychotherapies, regardless of their specific components, produce equivalent outcomes. The term originates from Lewis Carroll's novel Alice's Adventures in Wonderland, but the verdict originated from the work of Saul Rosenzweig in the 1930s. There are many controversies about this verdict because of the issues associated with testing the idea. The verdict is important, however, because of the many implications for psychology and the many different psychotherapies available for clients.

History

The “Dodo Bird Verdict” terminology was coined by Saul Rosenzweig in 1936 to illustrate the notion that all therapies are equally effective. Rosenzweig borrowed the phrase from Lewis Carroll's Alice's Adventures in Wonderland (1865), wherein a number of characters become wet and in order to dry themselves, the Dodo Bird decided to issue a competition: Everyone was to run around the lake until they were dry. Nobody cared to measure how far each person had run, nor how long. When they asked the Dodo who had won, he thought long and hard and then said "Everybody has won and all must have prizes." In the case of psychotherapies, the Dodo Bird Verdict maintains that all therapies are winners; they all produce equivalent outcomes.

According to the Dodo Bird Verdict, all psychotherapies, regardless of their specific components, produce equivalent patient health outcomes. The Dodo Bird debate took flight in 1975 when Lester Luborsky, Barton Singer and Lise Luborsky reported the results of one of the first comparative studies demonstrating few significant differences in the outcomes among different psychotherapies. This study spurred a plethora of new studies in both opposition and support of the Dodo Bird Verdict.

The Dodo Bird debate, in brief, is focused on whether or not the specific components of different treatments lead some treatments to outperform other treatments for specific disorders. Supporters of the Dodo Bird Verdict contend that all psychotherapies are equivalent because of "common factors" that are shared in all treatments (i.e., having a relationship with a therapist who is warm, respectful, and has high expectations for client success). In contrast, critics of the Dodo Bird Verdict would argue that the specific techniques used in different therapies are important, and all therapies do not produce equivalent outcomes for specific disorders.

Controversy

Support for the Dodo Bird Verdict

The common factors theory states that all therapies in psychology are equally effective because of the common factors they share. The only causal agents in treatment are the common factors and the specific techniques that are unique to treatment strategies are irrelevant.

There is research to support the common factors theory. One common factor is the client-therapist interaction, also known as the therapeutic alliance. A 1992 study by Lambert showed that nearly 40 percent of the improvement in psychotherapy is from these client-therapist variables. Other researchers have further analyzed the importance of client-therapist variables in treatment. They found that improvement in the patient was up to the patients' thought processes. Data shows that patients with more positive attitudes will have a better chance of experiencing clinical improvement, regardless of the therapist's actions. Furthermore, Wamphold et al. 2002, found that 7% of the variability in treatment outcome was due to the therapeutic alliance whereas 1% of the variability was due to a specific treatment. The therapists attitude is also a very important causal agent in positive patient change. Najavits and Strupp 1994, demonstrated that a positive, warm, caring, and genuine therapist generated statistically significant differences in patient outcome. Wampold et al. 2002, also found that nearly 70% of the variability in treatment outcome was due to the therapist’s attitude toward the efficacy of the treatment. Specific components of therapy are concluded to be relatively frivolous when compared with the more profound and directly patient affecting common factors.

Researchers have studied common factors in detail. Grencavage and Norcross 1990 identified 35 common factors in published sources. The identified common factors were categorized into five main groups: client characteristics, therapist qualities, change processes, treatment structures and relationship elements. Examples of some of the common factors included within these broad categories are persuasion, a healing setting, engagement, the use of rituals and techniques, suggestion, and emotional learning. Recently, Tracey et al. 2003, examined deeper relationships among the categories and common factors. They concluded that there are two dimensions of therapy: feeling and thinking. Within each of the two dimensions are three clusters: bond, information, and role.

Data provide evidence for the Dodo Bird Verdict. Generally speaking, common factors are responsible for patient improvement instead of any component of a specific treatment. Researchers such as Wampold and Luborsky continue to support this claim in recent articles published in the Behavioral Therapist. Wampold et al. 2010 refutes claims made by Siev et al. 2009 that Wampold 2009 made errors in research. Wampold et al. 2009, suggests that people need to, "accept the importance of the alliance and therapists and remain committed to developing and improving treatments." Wampold continues by saying that techniques could be beneficial in psychotherapy because they are the easiest variables to manipulate. These variables can act to change alliance and other common factors. Common factors can then be closely monitored and carefully distributed to the patient via the therapist.

Opposition to the Dodo Bird Verdict

The neutrality of this section is disputed. Relevant discussion may be found on the talk page. Please do not remove this message until conditions to do so are met. (September 2012) (Learn how and when to remove this message)

In rebuttal to the Dodo Bird Verdict, there is strong evidence that some treatments are in fact better for particular disorders when compared to other treatments. . Here, in contrast to the common factor theory, qualitative components of the therapy have statistically significant results. Luborsky, Singer, and Luborsky’s 1975 review on comparative outcomes of psychotherapies is one of the most well-known studies executed in the support of the Dodo Bird Verdict. They examined 100 comparative treatment studies and concluded that common factors across psychotherapies were pervasive. The outcomes of comparisons of different forms of psychotherapy were almost equivalent. However, although 30 years ago a general equivalency of the effect sizes of psychotherapies may have seemed reasonable, that is not at all the case today. With the improvement of research and the growth of psychology as a field in general, the treatments used by clinicians have evolved as well. Modern machinery helped producing more accurate results and reduced observational errors. Today, with the evolution of empirically supported treatments (ESTs) there is a greater emphasis on treatment integrity and specificity than what there had been early on in psychotherapy practices.

Empirically supported treatments are becoming more widely-used in treating psychosis. ESTs are “clearly specified psychological treatments shown to be efficacious in controlled research with a delineated population.” ESTs are developed from multiple control trial tests to evaluate which therapies yield the best results for specific disorders. These disorders include, but are not limited to, major depressive disorder, anxiety disorder, panic disorder, and OCD. According to the Ethics Code of the American Psychological Association (APA), psychologists and therapists have an obligation to avoid harming their clients in any way. ESTs are a major component in this movement. By using specific therapies clinically proven to improve certain disorders, therapists are avoiding further damage to their patients. However, supporting the idea of ESTs inevitably implies that some therapies are in fact more efficacious than others for particular disorders.

Probably the best evidence against the Dodo Bird Verdict is illustrated by the research done on anxiety disorders. Many studies have found ESTs to be beneficial when treating anxiety disorders, specifically cognitive behavioral therapy (CBT). CBT uses techniques from both cognitive therapy and behavioral therapy to modify maladaptive thoughts and behaviors. Numerous meta-analyses have shown that CBT yields significantly superior results in the treatment of psychological disorders, most notably, anxiety disorders. However, CBT also plays a positive role in treating depression, eating disorders, substance abuse disorders, and obsessive-compulsive disorder. In meta-analytic reviews, there is generally a larger effect size when CBT is used to treat anxiety disorders. The larger the effect size in a meta-analysis, the more evidence there is to support a claim or therapy. Recent studies show that when treating generalized anxiety disorder, CBT generated an effect size of 0.51, which indicates a medium effect. That is a much larger effect compared to supportive therapy and other treatments. Similarly, when treating social anxiety disorder, CBT produced an effect size of 0.62. Which again supports the evidence that CBT does in fact yield significantly better results than other therapies. Those supporting the Dodo Bird Verdict often use meta-analyses to compare multiple ESTs in order to illustrate that these treatments have no really significant differences. Because these treatments are already proven to work, comparing them to each other does show little variation. However psychology is made up of more therapies than just ESTs. When comparing a therapy like CBT to a treatment that does not meet EST criteria, the difference in effect size is obvious.

The primary premise for the Dodo Bird Verdict is that all therapies are equally efficacious and that any variation in therapeutic outcomes is most likely due to differences in therapist-client relationships. As discussed in this article, there are multiple meta-analyses supporting both sides of the debate. However, what is often overlooked in these meta-analyses is the notion that some therapies (in addition to not being effective) might actually be harmful. Indeed, “if psychotherapy is powerful enough to do good, it may be powerful enough to do harm,” as well. Some psychotherapies can be labeled as unhelpful, while others fall under the category of harmful. Unhelpful treatments are those that give no assistance while harmful therapies are actually damaging or dangerous to the patient. When identifying a “harmful” treatment it is important to note the distinction between “harm that can be caused by a disorder and harm that can be caused by the application of a treatment,” The negative outcomes of some psychotherapies are becoming an alarming reality in the field of clinical psychology. Studies have shown that individuals exhibited negative responses to treatment in some substance abuse work, and some types of grief therapy, and certain therapeutic techniques with trauma and PTSD patients. While those studies that support the Dodo Bird Verdict focus on the importance of building a client-therapist relationship, some studies have “identified a number of other relationship factors that may interfere with or negatively impact therapeutic change,”. The emerging evidence to the effect that there are possibly harmful psychotherapies is not only contradictory to the “all therapies are equal” stance of the Dodo Bird Verdict, but may also point out problems implicating the APA’s Code of Ethics. Many meta-analyses show that there are treatments that do yield more positive outcomes for specific disorders. However, proof that some treatments actually harm patients instead of helping them further erodes support for the Dodo Bird Verdict.

Issues in testing

A considerable amount of the controversy about the Dodo Bird Verdict relates to the meta-analytic methods used in research. These generate a lack of clear psychotherapeutic evidence for the support of either side in the debate. Meta-analytic studies have compared the effect sizes of different treatments, but have not been reliable in finding a consistent effect size. This could be because of several confounding variables. For example, many researchers are said to "have an agenda" when conducting meta-analyses, cherry-picking experiments they want to use in their study in order to produce the results they want. This pre-determined skewing results in unfair representations of the research. "Agenda"-based meta-analyses are confounded with the researcher's political, social, and economical opinions. Since psychologists are given the power to choose which studies are used in a meta-analysis, personal biases are involved, and the meta-analysis will produce biased results if the researcher is not careful in controlling for his or her own opinions.

Researchers opposing the Dodo Bird Verdict have found Dodo Bird supporters' meta-analyses to be "agenda"-based and highly subjective. Arguments have proposed that the specific meta-analysis the Dodo bird verdict is based on, (Luborsky, Singer, Luborsky 1975), could possibly produce misleading results because of the type of studies combined in the comparison. A startling 43% of studies that support the Dodo bird verdict are not sufficiently sensitive to detect differences in therapies even if they did exist. In Paul Crits-Christoph's review of Wampold et al. (1997) comparative study, a work that supported the Dodo bird verdict, Crists-Christoph concluded that out of the 114 articles used in the study, 79 of them involve similar comparative tests. Some meta-analyses constructed are not sensitive to the subtle distinctions between treatment effects, especially among comparative studies of highly similar treatments.

Dodo Bird supporting researchers have found anti-Dodo Bird supporters' research to also be "agenda"-based. For example, Wampold (2009), found Siev et al. (2007) study whose research for significance of CBT (Cognitive Behavior Therapy) versus RT (Rational Therapy) was resting on one experiment with an uncharacteristically large effect size (1.02) by Clark et al. (1994). Wampold found this effect size to be invalid because of the internal biases of the study. When this flawed experiment was removed from the analysis, the effect size was not statistically significant for the use of CBT over RT in panic disorder therapy. Against this research, in support of the anti-Dodo Bird Verdict, Chambless (2002) stated that “errors in data analysis, exclusion of research on many types of clients, faulty generalization to comparisons between therapies that have never been made, and erroneous sorts of treatments for all sorts of problems can be assumed to represent the difference between any two types of treatment for a given problem.”

Clearly, if meta-analyses are to be taken seriously, they need to be conducted carefully and objectively. In support of the anti-Dodo Bird side, Hunsley (2007) says that when “measurement quality is controlled for and when treatments are appropriately categorized, there is consistent evidence in both treatment outcome and comparative treatment research that cognitive and behavioral treatments are superior to other treatments for a wide range of conditions, in both adult and child samples.” This suggests that if and when variables are controlled, there is appreciable evidence for the superiority of cognitive and behavioral treatments.

Conclusion

Importance of the Verdict

The outcome of the Dodo Bird Debate has extremely important implications for the future of clinical psychology. For one, policymakers have to know how effective each existing psychotherapy is in order to be able to decide which therapies should be supported. This controversy may also lead governmental officials to cut funding for psychological treatments and other public health measures At the present, the public does not regard psychology as a hard science Although all of these studies evidence a uniform desire on the part of researchers to demonstrate that psychology too is evidence-based science, they may be doing more harm than good. By underlining each others' failings they detract from the scientific status of clinical psychology.

Perhaps the greatest illustration of the current state of the Dodo Bird Verdict is seen in the meta-analyses of Wampold and Barlow and the responses to it. In these meta-analyses, researchers on both sides point out the weaknesses and inconsistencies in their opponents' positions. Although both sides are trying to improve psychology in their respective ways, the disagreement about and lack of consistent evidence for the Dodo Bird Verdict may in fact be the cause of increased public doubt about the field. The conclusion of the debate could nationally dictate which therapists and which procedures will remain financially supported. For example, if the Dodo Bird Verdict is thought to be true regarding different psychotherapies, then many clinicians would feel free to use any therapy they see fit to employ. However, if the Dodo Bird Verdict is proven to be false, then clinicians would likely have to use empirically supported therapies when treating their clients. The debate continues.

References

1. Rosenzweig, Saul (1936). "Some implicit common factors in diverse methods of psychotherapy". American Journal of Orthopsychiatry. 6 (3): 412–415. doi:10.1111/j.1939-0025.1936.tb05248.x.
2. ^ Luborsky, L (1999). "The researcher's own therapeutic allegiances: a 'wild card' in comparisons of treatment efficacy". Clinical Psychology:Science and Practice. 6: 49–62. {{cite journal}}: |access-date= requires |url= (help)
3. ^ Luborsky, L; Singer, B; Luborsky, L (1975). "Is it true that 'everyone has won and all must have prizes?'". Archives of General Psychology 32: 995-1008.
4. Luborsky,L;Rosenthal, R; Diguer, L; Andrusyna, TP; Berman, JS; Levitt, JT; Seligman, DA; Krause, ED (2002). "The dodo bird verdict is alive and well-Mostly.". Clinical Psychology-Science and Practice 9: 2-12.
5. ^ Tarrier, N (2002). "Yes, cognitive behaviour therapy may be all you need". British Medical Journal 324: 291-292.
6. Frank, JD (1961). Persuasion and healing: A comparative study of psychotherapy.
7. ^ Wampold, BE (2007). "Psychotherapy: The humanistic (and effective) treatment". American Psychologist 62: 857-873.
8. Wampold, BE (2001). The great psychotherapy debate:models, methods, and findings.
9. Mahwah,NJ;Erlbaum,L;Horvath,AO;Bedi,RP (2002). The al-liance.
10. Martin,DJ;Garske,JP;Davis,MK (2000). "Relation of the therapeutic alliance with outcome and other variables: A meta- analytic review". Journal of Consulting and Clinical Psychology: 438-450.
11. Baldwin,SA;Wampold,BE;Imel,ZE (2007). "Untangling the alliance-outcome correlation: Exploring the relative importance of therapist and patient variability in the alliance". Journal of Consulting and Clinical Psychology 75: 842-852.
12. Klein,D.N., Schwartz,J.E., Santiago,N.J., Vivian,D., Vocisano,C., Castonguay,L.G., et al. (2003). "Therapeutic alliance in depres- sion treatment: Controlling for prior change and patient characteristics". Journal of Consulting and Clinical Psychology 71: 997-1006.
13. Lambert, M. (1992). Implications for outcome research for psychotherapy integration. In J. C. Norcross & M. R. Goldstein (Eds.), Handbook of psychotherapy integration (pp. 94–129). New York: Basic Books
14. Wampold, B. E. (1997). Methodological problems in identifying efficacious psychotherapies, Psychotherapy Research, 7, 21-43.
15. ^ Wampold, B. E., Minami, T., Baskin, T. W., & Tierney, S. C. (2002). A meta-(re)analysis of the effects of cognitive therapy versus “other therapies” for depression. Journal of Affective Disorders, 68, 159-165.
16. Wampold, B. E., Mondin, G. W., Moody, M., & Ahn, H. (1997). The flat earth as a metaphor for the evidence for uniform efficacy of bina fide psychotherapies: Reply to Crits- Christoph (1997) and Howard et al. (1997). Psychological Bulletin, 122, 226-230.
17. Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., & Ahn, H. (1997). A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, “All must have prizes.” Psychological Bulletin, 122, 203-215.
18. Wampold,B.E.,&Serlin,R.C.(2000).Thecon- sequences of ignoring a nested factor on mea- sures of effect size in analysis of variance. Psychological Methods, 5, 425-433.
19. NAJAVITS, L. M., & STRUPP, H. H. (1994). Differences in the effectiveness of psychodynamic therapies: A process-outcome study. Psychotherapy, 31(1), 114-123.
20. ^ Terence J. G. Tracey (2003): Concept Mapping of Therapeutic Common Factors, Psychotherapy Research, 13:4, 401-413
21. ^ Wampold, B. E., Imel, Z. E., and Miller, S. D. (2009). Barriers to the dissemination of empirically supported treatments: Matching evidence to messages. The Behavior Therapist, 32, 144-155.
22. Siev,J., Chambless, D.L. (2007). "Specificity of treatment effects: Cognitive therapy and re- laxation for generalized anxiety and panic disorders". Journal of Consulting and Clinical Psychology 75: 513-522.
23. Siev,J., Chambless, D.L. (2009). "The dodo bird, treatment technique, and disseminating empirically supported treatments". The Behavior Therapist 32: 69-75.
24. DeRubeis, R. J'Brotman, M.A., Gibbons, C.J. (2005). "A conceptual and methodological analysis of the nonspecifics argument". Clinical Psychology: Science and Practice 12: 174-183.
25. Clark,DM;Ehlers,A;Hackmann,A;McManus,F;Fennell,M;Grey,N;et al. (2006). "Cognitive therapy versus exposure and applied relaxation in social phobia: A randomized controlled trial". Journal of Consulting and Clinical Psychology 74: 568-578.
26. ^ Chambless, D.L., Hollon, S.D., (1998). "Defining empirically supported therapies". Journal of Consulting and Clinical Psychology 66: 7-18.
27. ^ Norcross, J.C. (1995). Dispelling the dodo bird verdict and the exclusivity myth in psychotherapy. Psychotherapy, 32, 500-504.
28. ^ Carroll, K.M., Roundsaville, B.J. (2010). Perhaps it is the dodo bird verdict that should be extinct. Addiction, 105, 18-20.
29. ^ Lilienfeld, O.S. (2007) Psychological treatments that cause harm. “Association for Psychological Science, 2,” 53-70.
30. American Psychiatric Association. (2002). Ethical principles of psychologists and code of conduct. “American Psychologist, 57”, 1067-1073
31. ^ Chambless, D.L., & Ollendick, T.H. (2001). Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685–716.
32. Barlow, D.H. (2004). Psycholgical treatments. "American Psycholgists, 59" 869-878
33. Treatment specificity for panic disorder: A reply to Wampold, Imel, and Miller (2009). Siev, Jedidiah, Huppert, Jonathan D., Chambless, D. The Behavior Therapist, Vol 33(1), Jan 2010, 12-14.
34. Watson, D., O’Hara, M.W., & Stuart, S. (2008). Hierarchical structures of affect and psychopathology and their implications for the classification of emotional disorders. Anxiety and. Anxiety and Depression, 25, 282–288.
35. Arch J.J., Craske M.G. (2009) First-line treatment: A critical appraisal of cognitive behavioral therapy developments and alternatives. "Psychiatric Clinic of North America. 32,"525–547
36. Hassett, A., Gevirtz, R., Nonpharmacologic treatment for fibromyalgia: Patient education, cognitive-behavioral therapy, relaxation techniques, and complementary and alternative medicine. “Rheumatic Disease Clinics of North America, 35,” 393-407
37. Beck A.T., (1964). Thinking and depression: Theory and therapy. “Arch Gen Psychiatry.10,” 561-571.
38. Butler A.C., Chapman J.E., Forman E.M., Beck A.T. (2006). The empirical status of cognitive-behavioral therapy: A review of meta-analyses. “Clinical Psychology Review, 26,”17–31
39. ^ Hofmann S.G., Smits J.A. (2008). Cognitive-behavioral therapy for adult anxiety disorders: a meta-analysis of randomized placebo-controlled trials. “Journal of Clinical Psychiatry. 69,” 621–32
40. ^ Otte, C (2011). Cognitive behavioral therapy in anxiety disorders: Current state of the evidence. “Dialogues Clin Neuroscience 13,” 413–21
41. Driessen E, Hollon S.D. (2010) Cognitive behavioral therapy for mood disorders: efficacy, moderators and mediators. “Psychiatric Clinics of North America. 33,” 537-555
42. Murphy R, Straebler S, Cooper Z, Fairburn C.G. (2010). Cognitive behavioral therapy for eating disorders. “Psychiatric Clinics of North America. 33,” 611-27
43. McHugh R.K., Hearon B.A., Otto M.W. (2010) Cognitive behavioral therapy for substance use disorders. “Psychiatric Clinics of North America. 33,” 511-25
44. Boileau B. (2011) A review of obsessive-compulsive disorder in children and adolescents. “Dialogues in Clinical Neuroscience.13,” 401-11
45. Norton P.J., Price E.C. (2007). A meta-analytic review of adult cognitive-behavioral treatment outcomes across the anxiety disorders. “The Journal of Nervous and Mental Disease.195,” 521–531
46. Lambert, M. J., & Ogles, B. M. ( 2004). The efficacy and effectiveness of psychotherapy. “Bergin and Garfield’s handbook of psychotherapy and behavior change 5,” 139– 193.
47. ^ Dimidjian, S., Hollon, D.S. (2010) How would we know if a particular form of psychotherapy was harmful? “American Psychology. 65,” 34-49
48. ^ Castonguay, G.L., Boswell, F.J., Constantino, J.M., Goldfried, R.M., Hill, E.C. (2010). Training implications of harmful effects of psychological treatments. “American Psychologist. 65,” 34-49
49. Bergin, A. E. ( 1971). The evaluation of therapeutic outcomes. ”Handbook of psychotherapy and behavior change “ 217– 270
50. Neimeyer, R. A. ( 2000). Searching for the meaning of meaning: Grief therapy and the process of reconstruction. “Death Studies, 24,” 541– 558
51. McNally, R. J., Bryant, R. A., & Ehlers, A. (2003). Does early psychological intervention promote recovery from posttraumatic stress? “Psychological Science in the Public Interest, 4,” 45– 79.
52. Nishith, P., Resick, P. A., & Griffin, M. G. ( 2002). Pattern of change in prolonged exposure and cognitive-processing therapy for female rape victims with posttraumatic stress disorder. “Journal of Consulting and Clinical Psychology, 70,” 880– 886
53. Gelso, C. J., Latts, M. G., Gomez, M. J., & Fassinger, R. E. ( 2002). Countertransference management and therapy outcome: An initial evaluation. “Journal of Clinical Psychology, 58,” 861– 867
54. Hill, C. E., Mahalik, J. R., & Thompson, B. J. ( 1989). Therapist self-disclosure. “Psychotherapy, 26,” 290– 295.
55. Norcross, J.C. (2002). Psychotherapy relationships that work: Therapist contributions and responsiveness to patients. New York: “Oxford University Press.”
56. ^ Hunsley, J., & Di Giulio, G. (2002). Dodo bird, phoenix, or urban legend? The question of psychotherapy equivalence. The Scientific Review of Mental Health Practice, 1, 11-22.
57. ^ Barlow, D. H. (2010). The dodo bird– again– and again. The Behavior Therapist, 33, 15–16.
58. Ferguson, C., & Brannick, M. (2012). Publication bias in psychological science: Prevalence, methods for identifying and controlling, and implications for the use of meta-analyses. "Psychological Methods, 17," 120–128
59. Reid, W.J. (1997). Evaluating the dodo bird verdict: Do all interventions have equivalent outcomes? Social Work Research, 21, 5-16.
60. Wilson G.T., Rachman S.J. (1983). Meta-analysis and the evaluation of psychotherapy outcome: Limitations and liabilities. Journal of Consulting and Clinical Psychology, 51, 54-64
61. ^ Kazdin, A.E., Bass, D. (1989). Power to detect difference between alternative treatments in comparative psychotherapy outcome research. Journal of Consulting and Clinical Psychology, 57, 138–147.
62. Crits-Christoph, (1997). Limitations of the Dodo Bird Verdict and the Role of Clinical Trials in Psychotherapy Research: Comment on Wampold et al. (1997). Psychological bulletin, 122, 216-220
63. Norcross, J.C., Rossi, J.S. (1994). Looking weakly in all the wrong places? Comment of Shapiro et al. (1994). Journal of Consulting and Clinical Psychology, 62, 535-538
64. Chambless, Dianne (2002). "(commentaries) Beware the Dodo Bird: the dangers of overgeneralization". Clinical Psychology: Science and Practice 9: 13–16. doi:10.1093/clipsy/9.1.13.
65. Beecham, J., Hallam, A., Knapp, M., Baines, B., Fenyo, A., & Ashbury, M. (1997). Costing care in hospital and in the community. In J. Leff (Ed.), Care in the community: Illusion or Reality? (pp. 93–108).
66. Chichester, England: Wiley. Glasgow, R.E., Klesges, L.M., Dzewaltowski, D.A., Bull, S.S., & Esta-brooks, P. (2004) The future of health behavior change research: What is needed to improve translation of research into health promotion practice? Annals of Behavioral Medicine, 27, 3–12.
67. Woolf, S.H. (2008). The meaning of translational research and why it matters. Journal of the American Medical Association, 299, 211– 213.
68. Hill, G.B. (2000). Archie Cochrane and his legacy. An internal challenge to physicians’ autonomy? Journal of Clinical Epidemiology, 53, 1189–1192.
69. Howard, K.I., Cornille, T.A., Lyons, J.S., Vessey, J.T., Lueger, R.J., & Saunders, S.M. (1996). Patterns of mental health service utiliza- tion. Archives of General Psychiatry, 53, 696–703.
70. Baker, B.T., McFall, M.R., Shoham, V. Current status and future prospects of clinical psychology. A Journal of the Association for Psychological Science, 9, 2, 67-103

• Misplaced Pages neutral point of view disputes from September 2012
• Figures of speech
• Psychotherapy