Skip to main content Back to Top


Study Questions Validity of HMO Quality Data

Kate Traynor

A report in the Sept. 25 Journal of the American Medical Association (JAMA) argues for mandatory reporting of health-care quality data by health maintenance organizations (HMOs).

The report questioned the usefulness of Health Plan Employer Data and Information (HEDIS) quality scores voluntarily supplied to the National Committee for Quality Assurance (NCQA). According to the report, HMOs with low HEDIS quality scores one year tend not to release their scores to the public the next year.

Specifically, HMOs with HEDIS scores in the lowest fourth in 1997 were 3.6 times as likely as plans in the top quartile to refuse to publicly report scores in 1998. Plans that scored in the lowest group in 1998 were 5.7 times as likely as top-scoring plans to keep scores private in 1999.

The report's authors accused HMOs of undermining the voluntary quality-monitoring process by making HEDIS data public only when the news is good. This "gaming of the system," the authors said, results in a data set that disproportionately represents the highest scorers. To get a true picture of the landscape, the authors called for mandatory quality-data reporting by all HMOs.

The report found that the use of HEDIS data to measure quality improvement is complicated by the ever-changing composition of NCQA's data set. Of the 329 HMOs that publicly disclosed their 1997 HEDIS scores, 73 declined to submit the data to NCQA for 1998 and 161 plans submitted data but did not allow it to be made public. The 1998 data set also included data from 130 newly participating plans that publicly disclosed their HEDIS scores.

Of the 292 plans that disclosed their 1998 quality data, 33 did not submit data for the next year, and 34 provided data to NCQA but did not allow public disclosure of the scores. Ninety-six new participants provided and disclosed quality data for 1999, for a total of 277 publicly disclosing plans that year.

The JAMA report was published a week after NCQA released its HMO quality report for 2001 (PDF), which contains data from 271 HMOs that allowed their HEDIS scores to be made public. According to a Reuters news report, NCQA calculated that about five of the HMOs that provided quality data for 2000 and allowed it to be made public refused to do so for 2001. NCQA President Margaret O'Kane portrayed the recent JAMA report as out of date, according to the news story.

During a Sept. 18 press conference that was held in Washington, D.C., to announce the 2001 data, O'Kane was enthusiastic about the number of health plans that allowed their HEDIS scores to appear in the report.

"About 65 percent of HMO and point-of-service plans, which cover 87 percent of all enrollees in the United States in that type of plan, gave us their results," O'Kane said. "We're very proud of the progress we've made both in getting more participation [and having] more public reporting."

O'Kane said that 85 percent of the health plans that submitted HEDIS data for 2001 allowed the data to be made public.

Bright spots in NCQA's latest report on commercial health-plan data included:

  • In all, 93 percent of patients received a beta-blocker after suffering a myocardial infarction in 2001, up from 85 percent in 1999. Thirteen organizations reported that every patient who had a myocardial infarction last year was prescribed a beta-blocker when discharged from the hospital.     
  • Seventy-five percent of children were vaccinated against chicken pox in 2001, compared with 64 percent in 1999.     
  • Eighty-one percent of patients with diabetes mellitus underwent testing for levels of hemoglobin A1c in 2001, a six-percentage-point increase from 1999. The percentage of patients with diabetes for whom a lipid profile was obtained rose from 69 percent in 1999 to 81 percent last year.

But HEDIS scores for antidepressant medication management declined among HMOs reporting data for 2001. In the 2001 report, 57 percent of patients with a depressive disorder did not receive appropriate treatment and follow-up, compared with 59 percent in 1999 (no data were available for 2000).

"We continue to see no progress on behavioral health," O'Kane said when describing the report. "There is definitely work to do on...effective use of medication management."