Skip to main content Back to Top


Utah Uses Controversial Data for Reporting Errors

Donna Young

In 1999, the Institute of Medicine (IOM), in its report To Err Is Human: Building a Safer Health System, called for the development of a nationwide mandatory reporting system that requires states to collect information about adverse events that result in death or serious harm.

IOM called for a reduction of errors by 50% within five years of the report’s release.

So far, 15 states have mandatory reporting programs, according to the National Academy for State Health Policy (NASHP) of Portland, Maine, a nonprofit organization that analyzes public policy.

Jill Rosenthal, NASHP program manager, said reporting systems vary from state to state in the types of data and the processes used in collecting information on adverse events and medical errors.

As a first step toward developing a reporting system, the state of Utah conducted a study on adverse events, including drug-related ones, using hospital discharge abstracts and the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) codes, including "external cause of injury" codes (E-codes). ICD-9-CM is the official system of assigning codes to diagnoses and procedures associated with hospitalization in the United States.

The state released its report, Adverse Events Related to Medical Care—Utah: 1995–1999, in June.

Gulzar H. Shah, programmer and analyst for Utah’s Department of Health, and one of the authors of the study, said his state recently introduced legislation to make adverse-event and medical-error reporting mandatory.

Correction. The above paragraph should read as follows: Gulzar H. Shah, programmer and analyst for Utah's Department of Health, and one of the authors of the study, said his state recently proposed administrative rules that would mandate the reporting of certain critical adverse events. The rules would also require hospitals to have programs to track and reduce the frequency of adverse drug events.

"Medical errors is a hot topic and a high priority for everyone right now since the Institute of Medicine report was published," Shah said. "In the absence of any other data, we used the ICD-9-CM codes. We needed some way of capturing the level of efficiency and some way to estimate trends and patterns."

The Utah report noted some limitations of using ICD-9-CM codes for tracking adverse events, including the inability to separate adverse events before hospitalization from those occurring during hospitalization, the inability to determine the clinical significance of the event, and the inability to distinguish variation in completeness of reporting from variation in the true occurrence of adverse events.

In the case of adverse drug events, E-codes may not be specific enough, according to NASHP.

NASHP uses the example of medication overdose: It could be reported as an intentional act by a patient or as a medical error. The codes, according to NASHP, are not precise enough to arrive at one correct conclusion.

NASHP also warns that the timeliness of availability of hospital data—running behind as much as three years in some states—restricts how useful the data might be in providing a real-time correlation.

Joyce A. Generali, director of Kansas University Medical Center’s Drug Information Center in Kansas City, said that, while it’s useful to take advantage of a system that is already in place, such as ICD-9-CM and E-codes, using the codes is fraught with limitations.

"The codes were never devised to track errors," she said. "It is an economically devised system. It’s totally dependent on documentation. Sometimes adverse events are not clearly documented. The biggest factor against using the codes to identify adverse events is it is incomplete data."

Generali said codes have little impact for management of adverse drug events since they are an "after-the-fact" system.

"Medication histories may not be available at the time adverse drug reactions are coded," she said. "Sometimes you have to comb through a patient’s medical and medication history to find out what was actually the cause of the adverse drug reaction. An adverse event does not necessarily mean that there was an error made."

But, Generali said, Utah’s use of the codes for its study was a "good start."

"It’s a first step, but they have to take into account what can and cannot be done with the data," she said. "It should be used as a supplement, not the end all."

But Patrice Spath, author of Error Reduction in Health Care: A Systems Approach to Improving Patient Safety, said she thinks states should steer clear altogether of using the codes for adverse-event reporting.

"The codes don’t necessarily indicate that an error has actually happened," Spath said. "It’s a numeric database without any lessons learned from it. For instance, a wrong medication could be given to someone but that person might not ever develop an adverse reaction, and nobody will ever know an error was made. There was no visible consequence, so it might not ever be labeled an error. The only way the error would be found is if someone went back through the patient’s medical chart and discovered that the wrong medication was given.

"Another example would be if a patient develops a rash. Is that from an error? It may not be. You can get outcome information using the codes. But you can’t assume that a patient’s good outcome is the result of no medical errors, just like you can’t assume that a bad outcome is the result of medical errors."

Spath advocates the gathering of adverse-event information directly from medical charts.

"But most researchers don’t want to take the time to do that," she said. "They like to use data that is easily accessible, like ICD-9-CM and E-codes. But that database was never meant to be used to make judgments about medical errors."

Spath said the word "error" is an illusive term.

"An injury can occur without an error happening," she said. "We should probably quit using the word ‘error’ in health care."

But Nancy Foster, of the Agency for Healthcare Research and Quality’s (AHRQ) Center for Quality Improvement and Patient Safety, said Utah’s study was an appropriate step toward the development of a reporting system—as long as it is recognized as a preliminary part of the process.

"The point of conducting experiments is to find out how much you can learn from them," she said.

But, she added, she agrees that while ICD-9-CM codes are widely used data sources, there are problems with relying on the data for adverse-event reporting.

"If a patient has surgery at one hospital and, unknowingly, a sponge is left in the patient, and then the patient goes to a different hospital to have the sponge removed, it may not necessarily ever get documented at the first hospital," she said.

Foster said AHRQ is funding a three-year project for 10–12 groups comprising state agencies, health care systems, and private industry organizations to develop and test reporting strategies and patient safety interventions.

The funding for the projects is in response to the 1999 IOM study.

"It’s something we can do as a federal agency to spur states to develop reporting systems," Foster said.

AHRQ released a request for applications for the projects in February and will announce the selected groups in mid-September.

Shah said his agency submitted an application for the project. He also said that part of the state’s purpose for conducting its study was to act as a basis for its application.

Utah’s application is a partnership with the state of Missouri, the University of Missouri in Columbia, and Intermountain Health Care, HealthInsight Inc., and the University of Utah in Salt Lake City, Shah said.

Correction. The above paragraph should include the Utah Hospital Association as a partner.

Foster said AHRQ is awarding up to $25 million in total costs to be shared among the recipients for the first year of the project. Funding beyond the initial budget period will depend upon annual progress reviews by AHRQ and the availability of funds, according to AHRQ.

Foster added that the goal of the project is, in three years, to have at least one model on which states can pattern their reporting systems.