Overview
The Outcome Rating Scale (ORS) and Child Outcome Rating Scale (CORS) are measures used to monitor children’s, young people and their families’ or carers’ feedback on therapeutic progress.
The ORS is a simple, four-item session-by-session measure designed to assess areas of life functioning known to change as a result of therapeutic intervention. These areas include: personal or symptom distress (measuring individual well-being); interpersonal well-being (measuring how well the user is getting along in intimate relationships); social role (measuring satisfaction with work/school and relationships outside of home); and overall well-being. The ORS translates these four dimensions of functioning into four visual analogue scales which are 10cm lines, with instructions to place a mark on each line with low estimate to the left and high to the right. The ORS is designed to be accessible to a child with a 13-year-old’s reading level, making it feasible for adolescents and adults.
The CORS was developed for children age 6–12. It has the same format as the ORS but with more child friendly language and smiley and frowning faces to facilitate the child’s understanding when completing the scales (Duncan et al., 2003).
For children 5 or under there is also the Young Child Outcome Rating Scale (YCORS), which can be a useful way of engaging young children regarding their assessment of how they are doing.
Terms of use
A license to use the ORS, CORS and YCORS in paper and pencil format is available for free to individual practitioners.
For group practices, agencies and behavioral health organizations please see the developers website for associated information: https://scott-d-miller-ph-d.myshopify.com/collections/performance-metrics/products/performance-metrics-licenses-for-the-ors-and-srs
All use of the measure is subject to the developer’s licensing conditions which can be found here: https://www.scottdmiller.com/group-licensing-agreement/
If you are planning to use these measures for the delivery and improvement of health and/or social care in the UK, a license to use them and to incorporate them into electronic systems can be obtained from NHS Digital.
The measure can be accessed from the developer’s website.
Suitability
The ORS can be completed by children aged 13-18 years old and the CORS can be completed by children aged 6-12 years old. Parents/carers can also complete the ORS/CORS. If the young person is 13 or over and fills out the ORS, the carer fills out the ORS on how they perceive the young person is doing. Similarly, if the young person is 12 and under and fills out the CORS, then the carer fills out the CORS on the young person.
Administration
The ORS/CORS/YCORS are straight forward to use with children and young people and facilitate collaborative, person-centred and feedback informed support/treatment. To get the best from these measures it is recommended to read the following articles:
Prepared for CYP-IAPT by: David C. Low, Norfolk & Suffolk NHS Foundation Trust, Scott D. Miller, International Center for Clinical Excellence (ICCE), Brigitte Squire, Cambridgeshire & Peterborough NHS Foundation Trust.
Also consult the developers website for a range of guidance and supporting materials.
It is also recommended that you take a look at this video made by young people and practitioners with support from CORC: https://www.youtube.com/watch?v=bU4n_WApnHc&t=215s
Working remotely
In response to the Covid-19 pandemic the developer shared the following guidance to using the ORS measures in remote ways: https://www.scottdmiller.com/getting-back-on-track-with-fit-deliberate-practice/
ICCE Certified Trainers provide instructions and examples for oral administration of the Session Rating Scale (SRS) which relates to the use of the ORS measures:
Scoring & Interpretation
If the paper version is being used, the guidance states that scoring is done with the young person/parent or carer using a ruler. Each of the four visual analogue scales is 10cm, so the score for each of the four visual analogue scales is the measurement length on the ruler (e.g. 3.3cm = score of 3.3) with 10 being the highest score for each scale. The score is written in the right margin, and the four scores added together for the overall score. The total possible score is 40. Next, each person’s overall score is plotted on a graph or entered into an electronic database to monitor the trajectory of progress.
Clinical cut-offs
The ORS (ages 13 and over) cut-off scores are:
Cut off for 13–17 year olds = 28
Cut off for 18 and over = 25
The CORS (ages 12 and under) cut-off scores are:
Child self-reporting* = 28 or 32
Carer reporting on child = 28
* Two cut-offs for the child-reported CORS have been published:
A cut-off of 32 was reported by Duncan et al. (2006) 1 and specified on the ORS/CORS Reliable Change Chart (Miller 2012). 2
A cut-off of 28 was reported by Duncan and Sparks (2017) 3 and discussed in a blog post by Duncan (2017).4
Psychometric properties
| Property | Definition | ORS |
| Internal consistency | Whether several items that propose to measure the same general construct produce similar score. |
Research on the ORS demonstrates high internal consistency, with coefficient alpha estimates of .93 (Miller, Duncan, Brown, Sparks, & Claud, 2003) and .97 (Bringhurst, Watson, Miller, & Duncan, 2006). Duncan, Sparks, Miller, Bohanske, and Claud (2006) also found robust internal consistency. |
| Test-retest reliability | Degree to which the same respondents have the same score after a period when a trait should not have changed. |
Research on the ORS test-retest reliability presents moderate to high estimates, .58 to .66 (Miller et al., 2003) and .80 to .84 (Bringhurst, Watson, Miller, & Duncan, 2006). |
| Concurrent validity | If a measure correlates well with a measure that has previously been validated. |
Pearson product-moment correlations between the ORS total score and the Outcome Questionnaire-45.2 (OQ-45.2) total score revealed a consistent pattern of moderately strong correlations between ORS and OQ-45.2 items, subscales and total scores, with an overall correlation between of total scores of .59 (Miller et al., 2003). Similarly, Binghurst et al. (2006) found overall correlations between the scores of these two measures ranging from .57 to .69. |
| Construct validity | Degree to which a test measures what it claims, or purports, to be measuring. |
Miller et al. (2003) found that the ORS was able to discriminate between clinical and non-clinical samples. Furthermore, factor analysis supported a one-factor model of the ORS, with items loading on a common factor shared with other similar outcome measures (Binghurst et al., 2006). |
Translation
A wide range of translations are available for the measure, accessed by the standard route from the developer’s website.
Useful resources
Additional guidelines and examples on how to administer, score and interpret the ORS can be found in the ORS manual, available at www.scottdmiller.com.
Information about the related Session Rating Scale is available here.
References
1 Duncan, B. L., Sparks, J., Miller, S. D., Bohanske, R., & Claud, D. (2006). Giving youth a voice: A preliminary study of the reliability and validity of a brief outcome measure for children, adolescents, and caretakers. Journal of Brief Therapy, 5, 66- 82.
2 Miller, S. (2012). ORS/CORS Reliable Change Chart. https://scottdmiller.com/wp-content/uploads/2016/09/ORS-Reliable-Change-Chart-Color.pdf
3 Duncan, B. & Sparks, J. (2017). The Partners for Change Outcome Management System. In: Lebow, J., Chambers, A., Breunlin, D. (eds) Encyclopedia of Couple and Family Therapy. Springer, Cham.
4 Duncan, B. (2017). The Ominous Clinical Cutoff and Data Accuracy. https://blog.betteroutcomesnow.com/2017/09/15/the-ominous-clinical-cutoff-and-data-accuracy
Anker, M.G., Duncan, B.L., & Sparks, J.A. (2009). Using client feedback to improve couple therapy outcomes: A randomized clinical trial in a naturalistic setting. Journal of Consulting and Clinical Psychology, 77, 692-704.
Duncan, B. L., Sparks, J., Miller, S. D., Bohanske, R., & Claud, D. (2006). Giving youth a voice: A preliminary study of the reliability and validity of a brief outcome measure for children, adolescents, and caretakers. Journal of Brief Therapy, 5, 66- 82.
Miller, S. D., Duncan, B. L., Brown, J., Sorrel, R., & Chalk, B. (2006). Using outcome to inform and improve treatment outcomes. Journal of Brief Therapy, 5, 5-22.
Bringhurst, D. L., Watson, C. W., Miller, S. D., Duncan, B. L., (2006). The reliability and validity of the Outcome Rating Scale: a replication study of a brief clinical measure. Journal of Brief Therapy, 5, 23–30.
Duncan, B. L., Miller, S. D., Sparks, J., Claud, D., Reynolds, L., Brown, J., Johnson, L., (2003). The Session Rating Scale: Preliminary psychometric properties of a “working” alliance measure. Journal of Brief Therapy, 3, 3–12.
Miller, S. D., Duncan, B. L., Brown, J., Sparks, J., & Claud, D. (2003). The Outcome Rating Scale: A preliminary study of the reliability, validity, and feasibility of a brief visual analog measure. Journal of Brief Therapy, 2, 91-100.
Reese, R. J., Norsworthy, L., & Rowlands, S. (2009a). Does a continuous feedback Version 17/01/12 27 system improve psychotherapy outcomes? Psychotherapy, 46, 418-431.
Reese, R. J., Usher, E., Bowman, D., Norsworthy, L., Halstead, J., Rowlands, S., & Chisholm, R. R. (2009b). Using client feedback in psychotherapy training: An analysis of its influence on supervision and counselor self-efficacy. Training and education in Professional Psychology, 3(3), 157-168.
Casey, P., Patalay, P., Deighton, J., Miller, S. D., & Wolpert, M. (2019). The Child Outcome Rating Scale: validating a four-item measure of psychosocial functioning in community and clinic samples of children aged 10–15. European Child & Adolescent Psychiatry, 29(8), 1089–1102.