Working with messy real-world data: the use of 'FUPS' data
The hope for big data in the NHS
It is increasingly argued that improving quality services should involve more scrutiny of routinely collected data (Keogh, 201). The aspiration is that data can be used to establish benchmarks for quality assurance and underpin the evaluation of quality improvement initiatives (Coulter, Locock, Ziebland & Calabrese, 2014).
Working with messy real-world data
CORC experience of child mental health outcome data is that the data are so flawed that they are hard to interpret. Indeed we argue that the data can be characterised as FUPS: flawed, uncertain, proximate and sparse. CORC believes that data might remain 'FUPS' for some time, and we need to be honest about what it can tell us to develop better quality data collection.
So what does FUPS mean?
The acronym FUPS refers to data that is:
Flawed - due to missing or erroneously recorded data
Uncertain - due to difference in how data items are rated and/ or variation in case mix
Proximate - in that they are always a proxy for an indication of the impact of the service provided
Sparse - in that even within complete datasets the low volume of cases within a given subgroup often limits the applicability of statistical inference.
Can we use it?
Respected researchers may argue that it is inappropriate to report findings based on FUPS data as they may lead to the wrong conclusions or be used in an unhelpful way. However, at CORC we believe that using this data is necessary as it may take many years before we have access to better quality data and that careful use of such data can help inform discussion amongst key stakeholders.
What next?
Our hope is that we can all consider the FUPS data we have, and this FUPS leaflet offers some useful guidance on how discussion about data can be facilitated.
CORC does, of course, see that it is important to have better data collection and higher quality data. However, in the meantime, we can start to examine the data we do have, with appropraite caution in the light of the FUPness of the data, and use the discussion to debate what outcomes can be achieved by those seeking help from child mental health services.
How can you get involved?
Read our leaflet about FUPS data, talk to colleagues and tell us the best ways you have made use of your FUPS data.