Research roundup: Monitoring treatment progress

While the specific goals of psychotherapy often vary for each patient, there is the underlying assumption that the process will be helpful to them. But how do you know that your client is progressing as expected? Research shows that clinicians often hold overly optimistic views of their patients’ treatment progress in relation to measured change (Walfish, McAlister, O’Donnell & Lambert, 2012).

One evidence-based solution is to use patient-rated quantitative measurements of progress. These tools offer continuous assessment of patient change and provide systematic feedback about patient response to treatment (e.g. Overington & Ionita, 2012).

The following research studies examine the efficacy of routine outcome monitoring, gather data regarding attitudes toward monitoring and feedback, and explore the challenges faced by practitioners who choose to integrate progress monitoring measures into their practice.

In addition to reviewing the following research summaries, psychologists are encouraged to explore the literature more completely to determine what may be useful to them in practice.

Lambert, M. J., Whipple, J. L., & Kleinstäuber, M. (2018). Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring. Psychotherapy, 55(4), 520–537. http://dx.doi.org/10.1037/pst0000167.

In this systematic review and meta-analysis, the authors sought to explore if tracking patient responses to treatment through routine outcome monitoring (ROM) practices improves overall outcomes for psychotherapy patients. ROM comprises routine measurement and tracking of patient progress using standardized self-report scales over the course of treatment. That information is shared with the clinician and ideally goes beyond what the clinician would have been able to observe and understand without it. The effects of two widely studied ROM systems—the Outcome Questionnaire System (OQ-System) and the Partners for Change Outcome Management System (PCOMS)—were considered in this review.

The OQ-System offers two components—a 45-item self-report measure that assesses three domains of functioning (symptoms of psychological disturbance, interpersonal problems, and social role functioning) used to predict treatment failure; and a clinical support tool (CST) composed of a problem-solving decision tree.

15 studies with 8,649 patients were included in the analyses for the OQ-System, of whom 1,958 were “not-on-track” (NOT) cases, meaning they were not progressing in treatment. The meta-analyses showed that the progress feedback intervention outperformed the psychotherapy as usually delivered in the total sample by a very small but statistically significant effect at the conclusion of treatment (effect size =.14). Of the 14 studies with sufficient power (i.e., appropriate sample sizes), 79% found a statistically significant effect for feedback compared with psychotherapy as usually delivered within a therapist’s caseload for NOT cases (effect size = .33). For the studies that added the CST to the progress feedback, a significant benefit of feedback plus CST was found on the total sample (effect size = .49).

Clinicians are likely to find implementing feedback using the OQ-System in routine care will enhance patient outcomes for all patients and are highly likely to reliably enhance patient outcomes for NOT patients.

The PCOMS utilizes two very brief scales: one with four items that focuses on mental health functioning (subjective well-being, interpersonal relations, social functioning, and overall sense of well-being) and a second four-item scale that is designed to assess the therapeutic alliance.

Nine studies with 2,272 patients were included in the analyses for the PCOMS. These studies focused on all patients, not just off-track patients. Of the nine studies, 67% reported a statistically significant difference between cases utilizing feedback and psychotherapy as usually delivered.

Clinicians are likely to find that implementing PCOMS will reliably improve patients’ outcomes beyond what is achieved for psychotherapy as usually delivered.

Jensen-Doss, A., Haimes, E.M.B., Smith, A.M., Lyon, A. R., Lewis, C.C., Stanick, C.F., & Hawley, K.M. (2018). Monitoring treatment progress and providing feedback is viewed favorably but rarely used in practice. Administration and Policy in Mental Health and Mental Health Services Research, 45(1), 48–61. http://dx.doi.org/10.1007/s10488-016-0763-0.

Despite the evidence that monitoring client progress and using feedback for clinical decision making enhances treatment outcomes, it is not clear how regularly these practices are used, nor are there psychometrically validated measures to assess attitudes toward monitoring and feedback in general. The authors of this study sought to validate two new attitudinal measures and examine the rates of use of standardized progress measures.

Approximately 500 mental health professionals completed survey items regarding demographics and professional and practice characteristics. Those who met the inclusion criteria (conducting or supervising intake assessments and/or therapy), also completed:

• The Monitoring and Feedback Attitudes Scale (MFA) to assess provider attitudes toward routine progress monitoring and feedback to clients about treatment progress
• Attitudes Toward Standardized Assessment Scales-Monitoring and Feedback (ASA-MF) adapted from the Attitudes Toward Standardized Assessment Scales (ASA; Jensen-Doss and Hawley 2010)
• Information regarding their usage of standardized progress measures

Factor analyses were conducted on both the MFA and ASA-MF. Items that did not load well were removed, resulting in two subscales with good internal consistency for the MFA: perception of general benefit associated with monitoring and feedback (MFA benefit, 10 items) and perception of harm associated with receiving negative feedback (MFA harm, four items); and three subscales with acceptable internal consistency on the ASA-MF: Clinical Utility (eight items), Treatment Planning (five items), and Practicality (five items).

On the MFA, providers reported positive attitudes toward gathering progress data and providing feedback to patients. Participants disagreed with the idea that feedback could harm the therapy alliance or make clients think their therapist is incompetent. Responses on the ASA-MF were more neutral, although respondents did strongly agree that standardized progress measures can help gather information that might not otherwise come up in session.

Regarding the use of standardized progress measures, only 5.2% reported using them every one to two sessions, 8.7% reported using them monthly, and 24.6% reported using them on a regular basis, but less than once a month. When asked how often they would prefer to use them, nearly 25% said they would like to gather frequent progress data, however, only 6.8% said they would prefer administering them every one to two sessions. 45% said they would prefer not to gather any progress data.

Ionita, G., Fitzpatrick, M., Tomaro, J., Chen, V.V., & Overington, L. (2016). Challenges of using progress monitoring measures: Insights from practicing clinicians. Journal of Counseling Psychology, 63(2), 173–182. http://dx.doi.org/10.1037/cou0000122.

The authors of this study sought to explore the challenges faced by practitioners who choose to integrate progress monitoring measures into their practice (independent or agency). Using a qualitative interview methodology, 25 licensed clinicians completed a survey on their usage of progress monitoring measures and then participated in open-ended, semi-structured interviews. Through a consensual qualitative research data analytic strategy, three domains represented the barriers experienced when using progress monitoring measures:

Technical concerns

• Concerns related to administering measures—additional work or time
• Dissatisfaction with specific characteristics of progress monitoring measures—length, validity, cost

Negative response from others

• Colleagues—measures challenge authority (experienced clinicians may be found to not be as effective as those with less seniority, clinician’s authority versus what client wants or believes), do not fit with their approach to practice, and difficulty adapting to change
• Clients—not seeing the purpose, discomfort with rating their therapist in their presence, bored by repetitiveness of completing same measure frequently
• Organization—measures believed to not fit with needs, lack of resources to implement

Therapists’ personal barriers

• Feeling uncomfortable or anxious about using measures—being evaluated, how to communicate with clients about the measures
• Lacking knowledge about progress monitoring measures

Additionally, strategies for overcoming those challenges were identified:

• Ensuring the fit of the measure—match to population, organization’s capabilities
• Increasing knowledge—researching and sharing the purpose and benefits of the measures
• Adapting one’s perspective—recognizing some clients like the measures, feedback is worth the discomfort, feedback was generally positive, and others had similar fears

Having an awareness of the potential barriers to implementing progress monitoring measures will offer clinicians a better understanding of what they may face and what strategies can be used to address these challenges.

Routinely monitoring patient progress during therapy is an important part of evidence-based practice in mental health (APA Presidential Task Force on Evidence-Based Practice, 2006). Results from the meta-analysis summarized above support the use of routinely monitoring the mental health of adult patients as they participate in psychotherapy (e.g. individual, couple, and/or group) using one of the several available programs, such as the OQ-System or the PCOMS. Real-time ROM feedback with alerts that identify at-risk cases assists clinicians in more accurately detecting patients who are worsening or who demonstrate less than expected improvement in psychotherapy.

Despite the demonstrated positive impact of using feedback monitoring tools and clinicians’ generally positive attitudes toward the usefulness of gathering progress data and providing feedback to patients, only a small minority of practitioners are currently using them at least once per month or even on a less regular basis. And many, when given the choice, would prefer not to use them at all. Findings from the qualitative study above help to identify some of the barriers clinicians found in implementing progress monitoring.

Clinicians who are interested in monitoring outcomes but have yet to do so can try several strategies to get started.

First, find tools and systems that work for your practice setting. Some take less than five to 10 minutes to administer. Also, continued advances in technology, including online scoring and feedback systems, have further reduced the amount of time it takes to administer, score, and interpret results.

Becoming comfortable at providing an explanation of the purpose and benefits of the measures may help to engender a positive response from patients. Some of this can be part of a well-implemented informed consent process.

Finally, connecting with experienced colleagues or through state associations and professional listservs could be helpful in allaying feelings of anxiety if you are uncertain regarding gathering progress data. An awareness that feelings of anxiety or discomfort are temporary may help clinicians to proceed.

On a larger scale, cultural changes at the organizational level may be needed for effective adoption. Allow for time, training and education regarding evidence that these systems do indeed add utility above and beyond simply checking in informally with patients every session.

Incorporating progress monitoring measures into training programs will likely lower anxiety or discomfort in using measures, allow practitioners to become familiar with the different characteristics of progress monitoring measures so they can choose ones with the best fit, and receive support and supervision to manage potentially negative responses.

Additionally, as more accountability from payers (e.g., Medicare and commercial insurance) is likely, figuring out progress monitoring that works for one's practice may help keep psychologists ahead of the game.

APA has resources for psychologists who wish to learn more about outcome monitoring programs.

• The Criteria for the Evaluation of Quality Improvement Programs and the Use of Quality Improvement Data page provides a set of criteria to be used by psychologists in evaluating quality improvement programs.
• The Mental and Behavioral Health Registry is an online database developed by APA for psychologists and other behavioral health practitioners. Psychologists can use this registry to track patient outcomes, meet quality reporting requirements, and keep their practice competitive.