Better data: completeness and validity

In the latest column from the Private Healthcare Information Network (PHIN), senior information services manager Peter Mills looks at the quality of the data being submitted to comply with the Competition and Markets Authority’s (CMA’s) requirements and inform patient decisions.

With over 600 hospitals and nearly 10,000 consultants submitting data to PHIN, we are able to provide a wealth of information on our website (phin.org.uk) to help patients make more informed healthcare decisions. However, the quality of data is just as important as the quantity.

We work closely with the hospitals and consultants submitting data to ensure high quality. This includes checking the data against 124 rules to understand how well providers are doing before we can give the data a ‘clean bill of health’. Examples of these rules include checking the format of the discharge date (often ensuring it is not in the future) or that the diagnosis code has a value as classified by the International Classification of Diseases 10th Revision (ICD-10), or that the ethnicity is within the defined list of values. These rules only check if the data is valid, and we also work closely with providers to ensure the accuracy of the data.

NHS England has similar rules when processing the data it receives from NHS acute hospitals via Secondary Uses Service (SUS). It checks things such as the patient’s identity (NHS number, postcode), characteristics (ethnicity, gender), admission and discharge details (how, where and when), care professional (GMC number and specialty identifiers) and clinical coding (diagnoses and procedures) are all valid codes. This helps ensure there is consistent data from the whole market.

What other data is collected and checked?

Other datasets collected and checked by PHIN, include adverse events, patient Satisfaction and experience, and Patient Reported Outcome Measures (PROMs) to enable us to report on areas such as never events, infection rates and responses to questionnaires for surgical and cosmetic procedures. With these we typically check that we have valid dates for the activity the questionnaire response relates to (for PROMs and feedback) along with ensuring the adverse events we report are valid, so that we do not over-report on infections for example, which are set out in a definitive list from the UK Health Security Agency (UKHSA).

We will soon be introducing changes around the mortality data we collect in our adverse events data set. This will allow us to improve the granularity of the measure we publish. Another upcoming change will be to check the primary procedure is correct and is an OPCS-4 code[1], in relation to the surgical treatment functions we see for admitted patients.

Data quality reporting is a key driver for PHIN in 2023 as it formed part of the industry-wide strategy agreed to deliver the CMA order. We will keep providers updated on upcoming changes.

Admitted Patient Care (APC) data submission

For this article, we looked at the results for five years’ worth of APC data from January 2018 to December 2022 focusing on validity, which is just one of the six ‘data quality dimensions’[2] that can be measured and analysed. It is closely related to completeness which only checks the presence of any value, particularly if the field is mandatory. Validity goes a step further than completeness and ensures that the data is correct and meets the requirements of the field in question. Accuracy is more difficult to measure than completeness or validity as it often relies on comparing fields to other fields or verification from an individual closely associated with the data. The APC data we receive generally has a very high validity percentage, with only the primary diagnosis (capturing symptoms as to why a patient requires treatment) and treatment function code (the service within a hospital that will manage the care of a patient) dropping below 99%; and they still remain in the 90s at 93.2% and 98.7% respectively.

Volume and Length of Stay measures from APC dataset (2018-2022)

Table shows which APC fields that are used to construct a volume and length of stay measure have a high validity rate between 2018 and 2022.

Why are diagnosis and procedure clinical coding so important?

The following chart shows the validity of the diagnosis and procedure clinical coding data collected by PHIN over the past five years.

Improvements in APC data validity

To dig deeper into the APC data submissions, we set out to answer the following questions by looking into how many records raised no validation issues (interested in which providers of private healthcare and how many records the quality relates to):

1. has improved over time
2. has improved over a single year significantly
3. has remained consistently good
4. has regressed (if at all)?

Improved over time

The data shows that the number of records submitted with no validation issues has increased year-on-year since 2018. This suggests that the submitting organisations have implemented processes and procedures to ensure data is captured and submitted in accordance with the Private Healthcare Market Order (the Order).

Providers improving year-on-year

Providers and total count of APC records supplied to us between 2018 and 2022

London Welbeck Hospital for example, has supplied 8,180 records since 2018 and has been working diligently over the years alongside PHIN’s Informatics and Engagement Teams. The number of records with no validation issues are improving year on year.

Number of years with >90% improvement in validity

We looked at how many years a provider has improved in validity on the previous year (where the threshold is greater than 90% improvement compared to the previous year, for records which raise no validation issues). We looked at data supply for the past five years.

This section is only really applicable to smaller providers which had low validity and admissions so that the increases look more significant. For example, Spencer Private Hospitals has been working closely with PHIN over the years to ensure all its data is in good standing.

Top five providers for >90% improvement

This table shows providers with the greatest improvement in APC records validity between 2018 and 2022.

Consistently great or good

Over the past five years, Curis Health Care has submitted 5,275 records with >95% of records having no validation issues. Over the same period, New Victoria submitted nearly 25,000 records and also scored well at >90% of records having no issues. Over the past four years, James Paget University Hospitals NHS Foundation Trust also scores highly.

No provider has regressed in the five-year period.

Consistently good recently (past 3 years 2020-2022)

Here we looked at whether each year has a high percentage (>90%) of records that have no validation issues.

Top 5 providers for consistently providing high percentage of records with no validation issues

This table shows providers and number of APC records supplied to PHIN between 2018 and 2022.

Curis Health Care has been setting an excellent standard in terms of the admitted patient care data submitted to us and New Victoria is hot on its heels.

Great progress

Providers are making great progress in supplying the required information and to high standards. PHIN is grateful to everyone for their huge efforts and delighted to be able to give patients a more informed view of the private healthcare sector.

We are here to help anyone who has any questions about what needs to be submitted and when.

[1] A statistical classification of interventions and procedures undertaken in the National Health Service (NHS) reflecting current clinical practice.

[2] Accuracy, completeness, uniqueness, consistency, timeliness, and validity https://www.gov.uk/government/news/meet-the-data-quality-dimensions