Data Validation Process In Summary

Measurement is about selecting what is to be measured, selecting and testing the
measure, collecting the data, validating the data, and using the data for improvement.

Validating the data is an important tool for understanding the quality of the quality data which is reliable, accurate, and defensible data that has been validated, for establishing the level of confidence decision makers can have in using data and in their implications for clinical practice.

An example of performance measurement is when an area for improvement in structure, process, and/or outcome is identified, new guidelines for patient care and safety are usually developed by the hospital using the data which had been selected, tested, collected, validated for patient care and safety improvement. This change process is normally managed by the hospital and include key stakeholders (e.g., clinicians) affected by the change.

An example of data validation when Health Information Management (HIM) / Medical Records (MR) practitioners who are generally specialised or experts is in disease coding may be involved, is when they provide advice in disease coding validation studies to determine staff training needs.

To ensure that a sample is valid when evaluating performance, it is critical to always determine an appropriate sample size ie. the number of subjects to choose, a procedure to ensure that your sample is representative of the population i.e the degree to which the subjects are similar to those in the intended use, and also determine the types of data to be used (administrative or clinical).

Well, you need to sample so as to try to get one that represents the population as closely as possible. This is because we rarely have enough time and money to look at the entire group of people that we are interested in (for example, the population of everyone attending a clinic at a particular hospital).

In trying to getting a valid sample, let us assume you had limited money, you cannot
study the target population as a whole. By all means do select a small sample size but when you choose a small sample size, there is always a higher risk of sampling error being present, for example when you could only choose only two patients out of the population of 30.

Unclear data definitions and inconsistent coding of data are reasons when data elements are found not to be the same. It is vital to have a list of codes with their definitions that you are going to be using throughout the collection of data. For example, if you are coding ward clerk as 1 and charge nurse as 2, it is important to ensure that you have used the same codes throughout the process of entering data into the dataset. In data validation, It is important to make corrective actions when inconsistent coding of data is found. However, if you do decide to change some data codes, it would be wise to note any changes as you progress.

The chart below characterises the process of data validation (by clicking on the chart below, it will open in a new tab of your current window, and by clicking on the image in this new tab, you can view a larger view of the chart).

Data Validation Process

Data validation to ensure that good, useful data have been collected

Anyone who deals with data, will know that data is first acquired (collected) and verified (validated) before data input. Data input is then processed or managed which includes data storage, data classification, data update, and data computation. Data output is when the data input and processed or managed is retrieved and data is presented in a meaningful way.

Data acquisition (collection), data verification (validation), data classification, data storage, data update, data computation, data retrieval and data presentation are the eight elements which make up the three phases when we deal with data, that is the data input phase, the data management or processing phase and lastly, the data output phase.

Data are the raw materials that involves both the generation and the collection of accurate, timely, and relevant data through reliable measurements that ensures good, useful data have been collected.

Good, useful data involves using an internal data validation process in the authentication and validation of gathered data from authoritative, valid, and reliable data sources. It is important to consider applying the garbage in garbage out (GIGO) principle in collecting valid data.

If your hospital is implementing or has already begun a quality improvement program for example the Joint Commission International (JCI) hospital accreditation program, the quality of your hospital’s quality improvement program  is only as valid as the data that you have collected through reliable measurements.

When using data for improvement and for establishing the level of confidence decision makers can have in the data when implementing or starting a quality improvement program, JCI (2011, pg. 156) recommends data validation in these following circumstances :

  • a new measure is implemented (in particular, those clinical measures that are intended to help an
  • hospital evaluate and improve an important clinical process or outcome);
  • data will be made public on the hospital’s Web site or in other ways;
  • a change has been made to an existing measure, such as the data collection tools have changed or the
  • data abstraction process or abstractor has changed;
  • the data resulting from an existing measure have changed in an unexplainable way;
  • the data source has changed, such as when part of the patient record has been turned into an electronic
  • format and thus the data source is now both electronic and paper; or
  • the subject of the data collection has changed, such as changes in average age of patients, comorbidities,
  • research protocol alterations, new practice guidelines implemented, or new technologies and treatment methodologies introduced.

JCI (2011, pg. 157) also recommends the following essential elements of a credible data validation process as an important tool for understanding the quality of the quality data:

  1. re-collecting the data by a second person not involved in the original data collection
  2. using a statistically valid sample of records, cases, and other data; a 100% sample would only be needed when the number of records, cases, or other data is very small
  3. comparing the original data with the re-collected data
  4. calculating the accuracy by dividing the number of data elements found to be the same by the total number of data elements and multiplying that total by 100. A 90% accuracy level is a good benchmark
  5. when data elements are found not to be the same, noting the reasons (for example, unclear data definitions) and taking corrective action
  6. collecting a new sample after all corrective actions have been implemented to ensure the actions resulted in the desired accuracy level

Health Information Management (HIM) / Medical Records (MR) practitioners do take note that ff your hospital is a hospital which is already JCI accredited or seeking JCI accreditation status or undergoing re-survey for JCI accreditation status, then it must integrate data validation into its quality management and improvement processes, has an internal data validation process that includes (1) through (6) above, and the data validation process must include at least the measures selected as required in Standard QPS.3.1 when “The organization’s leaders identify key measures for each of the organization’s clinical structures, processes, and outcomes.” Such identified key measures is usually integrated as an ongoing standardised process to evaluate the quality and safety of the patient services provided by each medical staff member as required by the JCI Standard SQE.11 In other words, each of the hospital’s clinical structures, processes, and outcomes provided by each medical staff member are evaluated, and conclusions drawn from in-depth analysis of known complications of clinical structures, processes, and outcomes as applicable which are in turn used for all corrective actions to be implemented.

References:

  1. Joint Commission International, 2010, Joint Commission International Accreditation Standards For Hospitals, 4th edn, JCI, USA
  2. Joseph, T & Payton, FC, 2010, Adaptive health management information systems : concepts, cases, & practical applications, 3rd edn, Jones and Bartlett Publishers, Sudbury, MA, USA