News and Blogs

Routine Outcome Monitoring in Child and Youth Mental Health: What does the latest research tell us?

Dr Jenna Jacob

Use of routine outcome monitoring

Routine outcome monitoring (ROM) provides information to practitioners and others, usually in the form of questionnaires, to evidence the impact of mental health and wellbeing interventions. For many years we have promoted the use of outcome measures in a way that is also meaningful to practitioners and young people and supports them in the work they do together. Over time we have seen a shift away from viewing outcome measurement as a tick box exercise, and increasingly see excellent examples of outcome measures being embedded in a way that helps track and improve outcomes for children and young people.

Challenges, solutions and what the evidence says works

Whilst ROM is widely used in young people’s mental health services, we are well aware of the challenges to embedding this in practice, e.g., limited resources, unfit IT systems, lack of trust in the measurement tools, lack of “buy-in” from clinicians (1–5) or caution from young people about whether measures can fully capture their needs(6). However, research also finds that using measures can improve on clinical judgement in identifying when young people will decline from an expected trajectory of change (7–9); and that ROM can provide clinical information that may have otherwise be missed (10,11). There is also some great emerging evidence which suggests that practitioners and young people don’t always align on what they think the most important areas of change for young people are, which really highlights the importance of checking in about outcomes together (12).

We regularly work with our CORC members to help them to overcome some of the challenges of implementing ROM, and often share with them research which demonstrates the unique ways in which the use of measures can impact their work –and improve outcomes for young people. Some of these strategies have been published, e.g., from the CORC perspective: (3,13), and more general learning from work with adults (2). We felt like it was time to revisit that research and refresh our knowledge summary.

In fairly recent reviews of the evidence relating to the impact of using outcome measures, better outcomes were reported when feedback was provided to young people (18) and to both the practitioner and young person (19,20). We are not aware of any more recent reviews of the impact of ROM, rather experiences of their use (e.g., (6)). It’s worth noting that there are reviews of the implementation of ROM for adults that we could learn from, but I have chosen to keep this focused on young people. Having said that, the need for more evidence in this area for adults and young people has been highlighted (21,22). In terms of young people’s experiences of ROM, when used as part of collaborative practice and shared decision-making, young people have said that the use of measures is a helpful tool for self-reflection and increased communication with their practitioner (6,23). ROM has been described by young people as a way for them to feel heard, valued and understood (23–26) and an important vehicle for a mutual understanding and shared ways forward (26,27). Research also tells us that ROM may be linked to higher levels of retention in services (28,29).

Given the above, it is important to clarify the difference between 1) completing measures, and 2) using measures as part of feedback. The latter involves talking to young people about their responses, and integrating that into the work. Fundamental to these discussions is a focus on areas of outcome tracking that are important to the young person (14–17). There are several structured feedback systems available that can help to facilitate conversations about measures and progress, between practitioners, and with young people.

There is an ethical consideration to discuss outcome information with young people. It is important to flag that not everyone who receives specialist input for mental health difficulties wants to actively track and talk about outcome measures (31), and so conversations about client preferences are incredibly important, balancing these with service requirements. The meaningful use of ROM is what we discuss most with CORC members. We continue to advise the use of multiple sources of outcome information; standardised, idiographic and from different reporters, to ensure that a breadth of information is captured and can be considered for use with children and young people.

What’s next?

My co-Research Lead, Julian, and I are Guest Editors for a Special Issue Monitoring and Measurement in Child and Adolescent Mental Health for the International Journal of Environmental Research and Public Health. We have written an Editorial (30), which covers some of the topics I have raised here, and introduces the special issue. We are inviting a wide range of research papers to be submitted (deadline for submissions is 15^th July, more info is here), focused on a multifaceted exploration of ROM. We hope it helps to promote reflection on all elements of ROM, retaining at the heart a considered focus on young people.

There doesn’t seem to be too much new research in the young person’s field focused on the impact of using outcome measures, with the latest research being the reviews from 2016-2018 (18–20). It does seem as though attention has turned to specific elements of outcome measures use, as outlined here, and the exploration of specific measures. There is also much energy around exploring the impact of more person-centred measures, particularly goal-based outcome measures. In my next blog, I’ll be writing about what type of measure to use, what the data says, and bias in measurement, so watch this space.

 

 

References

1. Batty MJ, Moldavsky M, Foroushani PS, Pass S, Marriott M, Sayal K, et al. Implementing routine outcome measures in child and adolescent mental health services: From present to future practice. Child Adolesc Ment Health. 2013;
2. Boswell JF, Kraus DR, Miller SD, Lambert MJ. Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions. Psychother Res [Internet]. 2015;25(1):6–19. Available from: http://dx.doi.org/10.1080/10503307.2013.817696
3. Fleming I, Jones M, Bradley J, Wolpert M. Learning from a Learning Collaboration: The CORC Approach to Combining Research, Evaluation and Practice in Child Mental Health. Adm Policy Ment Heal Ment Heal Serv Res [Internet]. 2016;43(3):297–301. Available from: http://dx.doi.org/10.1007/s10488-014-0592-y
4. Hall CL, Moldavsky M, Taylor J, Sayal K, Marriott M, Batty MJ, et al. Implementation of routine outcome measurement in child and adolescent mental health services in the United Kingdom: A critical perspective. Eur Child Adolesc Psychiatry. 2014;23(4):239–42.
5. Langer, D. A. & Jensen-Doss A. Shared Decision-Making in Youth Mental Health Care: Using the Evidence to Plan Treatments Collaboratively. J Clin Child Adolesc Psychol. 2018;47(5):821–31.
6. Solstad SM, Castonguay LG, Moltu C. Patients’ experiences with routine outcome monitoring and clinical feedback systems: A systematic review and synthesis of qualitative empirical literature. Psychother Res. 2019;29(2):157–70.
7. de Jong K, van Sluis P, Nugter MA, Heiser WJ, Spinhoven P. Understanding the differential impact of outcome monitoring: Therapist variables that moderate feedback effects in a randomized clinical trial. Psychother Res. 2012;22(4):464–74.
8. Hannan C, Lambert MJ, Harmon C, Nielsen SL, Smart DW, Shimokawa K, et al. A lab test and algorithms for identifying clients at risk for treatment failure. J Clin Psychol. 2005;61(2):155–63.
9. Lambert M. “Yes, it is time for clinicians to routinely monitor treatment outcome.” In: B. L. Duncan, S. D. Miller, B. E. Wampold & MAH, editor. The heart and soul of change: Delivering what works in therapy. American Psychological Association; 2010. p. 239–66.
10. Worthen, V. E., & Lambert MJ. Outcome oriented supervision: Advantages of adding systematic client tracking to supportive consultations. Couns Psychother Res. 2007;7(1):48–53.
11. Emanuel, R., Catty, J., Anscombe, E., Cantle, A., & Muller H. Implementing an aim-based outcome measure in a psychoanalytic child psychotherapy service: Insights, experiences and evidence. Clin Child Psychol Psychiatry. 2014;19(2):169–83.
12. Krause KR, Edbrooke-Childs J, Singleton R, Wolpert M. Are We Comparing Apples with Oranges? Assessing Improvement Across Symptoms, Functioning, and Goal Progress for Adolescent Anxiety and Depression. Child Psychiatry Hum Dev [Internet]. 2021;(0123456789). Available from: https://doi.org/10.1007/s10578-021-01149-y
13. Edbrooke-Childs J, Gondek D, Whelan I, Jacob J, Barnard M, Gleeson H, et al. Evaluating interventions for children, young people and families: Theory, evidence, policy, and lessons learnt. Handbook of Social Policy Evaluation. 2017.
14. Hickie IB, Scott EM, Cross SP, Iorfino F, Davenport TA, Guastella AJ, et al. Right care, first time: a highly personalised and measurement-based care model to manage youth mental health. Med J Aust. 2019;211(S9):S3–46.
15. Jacob J, Edbrooke-Childs J, Law D, Wolpert M. Measuring what matters to patients: Using goal content to inform measure choice and development. Clin Child Psychol Psychiatry. 2015;22(2).
16. Krause KR, Bear HA, Edbrooke-Childs J, Wolpert M. Review: What Outcomes Count? A Review of Outcomes Measured for Adolescent Depression Between 2007 and 2017. J Am Acad Child Adolesc Psychiatry [Internet]. 2019;58(1):61–71. Available from: https://doi.org/10.1016/j.jaac.2018.07.893
17. Lavik KO, Veseth M, Frøysa H, Binder PE, Moltu C. What are “good outcomes” for adolescents in public mental health settings? Int J Ment Health Syst [Internet]. 2018;12(1):1–10. Available from: https://doi.org/10.1186/s13033-018-0183-5
18. Tam HE, Ronan K. The application of a feedback-informed approach in psychological service with youth: Systematic review and meta-analysis. Clin Psychol Rev [Internet]. 2017;55(January):41–55. Available from: http://dx.doi.org/10.1016/j.cpr.2017.04.005
19. Gondek D, Edbrooke-Childs J, Fink E, Deighton J, Wolpert M. Feedback from Outcome Measures and Treatment Effectiveness, Treatment Efficiency, and Collaborative Practice: A Systematic Review. Adm Policy Ment Heal Ment Heal Serv Res. 2016;43(3):325–43.
20. Lambert MJ, Whipple JL KM. Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring. Psychotherapy. 2018;55(4):520–37.
21. Bergman, H., Kornør, H., Nikolakopoulou, A., Hanssen‐Bauer, K., Soares‐Weiser, K., Tollefsen, T. K., & Bjørndal A. Client feedback in psychological therapy for children and adolescents with mental health problems. Cochrane Database Syst Rev. 2018;8.
22. Kendrick T, Moore M, Gilbody S, Churchill R, Stuart B, El-Gohary M. Routine use of patient reported outcome measures (PROMs) for improving treatment of common mental health disorders in adults. Cochrane Database Syst Rev. 2014;2014(6).
23. Feltham, A., Martin, K., Walker, L. & Harris L. Using goals in therapy: The perspective of people with lived experience. In: M. Cooper & DL, editor. Working with goals in psychotherapy and counselling. Oxford: Oxford University Press; 2018. p. 73–85.
24. Greenhalgh, T., Humphrey, C., Hughes, J., Macfarlane, F., Butler, C., & Pawson RAY. How do you modernize a health service? A realist evaluation of whole‐scale transformation in London. Milbank Q. 2009;87(2):391–416.
25. Law, D., & Wolpert M. Guide to using outcomes and feedback tools with children, young people and families. Formally known as the COOP document. London: CAMHS Press; 2014.
26. Martin K. A critical realist study of shared decision-making in young people’s mental health inpatient units. 2019;
27. Bennett LR. Adolescent Depression: Meeting Therapeutic Challenges Through an Integrated Narrative Approach. J Child Adolesc Psychiatr Nurs. 2012;25(4):184–94.
28. Cairns, A. J., Kavanagh, D. J., Dark, F., & McPhail SM. Goal setting improves retention in youth mental health: a cross-sectional analysis. Child Adolesc Psychiatry Ment Health. 2019;1–8.
29. Miller, S. D., Duncan, B. L., Brown, J., Sorrell, R., & Chalk MB. Using formal client feedback to improve retention and outcome: Making ongoing, real-time assessment feasible. J Br Ther. 2006;5(1):5–22.
30. Jacob, J.; Edbrooke-Childs J. Monitoring and Measurement in Child and Adolescent Mental Health: It’s about More than Just Symptoms. Int J Environ Res Public Heal. 2022;9(4616).
31. Jacob J. Collaborative goal setting: what works for whom in therapeutic relationships? [Internet]. 2021. Available from: https://www.corc.uk.net/news-blogs/collaborative-goal-setting-what-works-for-whom-in-therapeutic-relationships/