Findings:
Data Quality
The Texas Department of Health (TDH) does not discuss data quality problems in its Summary Report. However, both A&M and THQA focused a great deal of attention on the adequacy of the data used to evaluate service received through the managed care pilots.
There are several types of information necessary to assess access and quality of service provided to Medicaid clients: encounter data, utilization management data, focused quality of care studies, and medical records. THQA formally audited each of these data types, while A&M reported numerous problems with access to the data necessary to complete their research. "Considerable energy should be invested," said the A&M team, "in improving the management information systems and databases that provide numerous opportunities for learning and oversight. This evaluation suffered immensely from the databases from which the evaluation team had to work."
Consumers Union reviewed the data audit performed by THQA (Encounter Report), and found that the significant problems with data about the services Medicaid clients are getting undermines any effort to establish whether access and quality have improved or declined in managed care areas. Probably the fundamental problem lies with the collection of "encounter" data.
Encounter data is the record of each service received or performed for a patient. It is the primary data necessary to know whether patients are being served and how often. A patient who visits the doctor has one "encounter." If the same patient revisits the doctor a week later for a follow-up, this is another "encounter." FFS Medicaid creates accurate encounter data, because providers bill for each service. HMOs find encounter data notoriously hard to collect, because doctors who are paid on a capitation basis (a certain amount per member per month) or hospitals paid on a per diem basis have little incentive to adequately report to the HMO each service they provide.
Prior to managed care, the state could track the amount and kind of service provided to children at every age, pregnant women, older adults, men or women. Although the NHIC system did not readily produce non-standard reports, the state used the data to project program needs and determine which populations were not adequately served.5 Today, the FFS encounter data excludes information about the state's largest urban areas now under managed care, and the managed care program has not been able to create new encounter data to fill the gap.
According to THQA, only 20.3 percent of "encounters" found in patient medical records were recorded in the HMO administrative database. In the Travis service area, only 7.7 percent of encounters appeared in both the medical record and the database. Auditors determined that they could probably identify some additional partial record matches (records could be found for the same member on the same date of service but did not match for type of service provided, for example), and if you add those encounters, HMOs can find a total of 30.2 percent of medical record encounters in their database. PCA in Bexar had less than one percent of medical record encounters in its database, even using partial matches. The plan with the best data audit in the program, Community First, had less than half of its medical record encounters in its administrative database (Encounter Report, pp. 16-17).
The most common problem (42.5 percent of errors) was a "false negative"-a service that was provided and appears in the patient's medical records but did not appear in the HMO database. This indicates that Medicaid enrollees are getting more services than we know about (Encounter Report, p. 16, 25).
However, the degree to which existing data underestimates services is reduced significantly by the large number of "false positives"-services that appear in the administrative data but not in the patient's medical record (which would confirm that the services were actually received). The false positive rate was 23.8 percent across all plans. THQA attributed the high "false positive" rate in part to coding errors (Encounter Report, p. 18, 25).
To muddy the picture even more, TDH relies upon reports from NHIC, the claims administrator, to get a full picture of the Medicaid program. Managed care organizations submit data to NHIC, which edits the data for form and format problems. Any data that doesn't pass the edit is returned to the managed care organization. After editing and correction, NHIC produces a "history" file for use.
THQA compared the data submitted by the HMOs to the final NHIC history file, expecting the latter to be an audited version of the former. Instead, they found that more than a third of encounters in the final "history" file did not match encounters in the HMO data. Looking at these "extra" encounters along with the "false negatives" (encounters in the HMO file but not in the final NHIC file), and matching them generously, auditors determined that about 80 percent of the encounters in managed care organization administrative data could be found in the state's final "history" report. The other 20 percent could not (Encounter Report, p. 20-21).
More important to the state's quality assurance efforts, only about 10 percent of encounters in the patient medical record are identifiable in the state's "history" data. And almost two-thirds of the encounters recorded in the state "history" files could not be found in the available medical records (Encounter Report, p. 21, 23)
The clear conclusion from this audit is that neither the state nor the managed care organizations know, based on reliable data, the services being provided under the plans. THQA recommended that contract renewal should be linked to the accurate submission of encounter data, and the state should issue a unique provider ID number to be used by each plan to clear up some data coding problems.
Last year, TDH required managed care organizations to change the way they submitted data, in preparation for the completion of Compass 2000, an updated version of NHIC's existing claims database. Managed care organizations will be required to make a final set of changes to the data reporting formats this summer. The new state database should produce reports more flexibly than before, and it attempts to identify all managed care encounters as "claims" even if no billing statement is ever generated. These changes, however, do not address under reporting by doctors, or data coding problems within and among plans. It will take a concerted and long term effort backed by contract enforcement to give Texas the data it needs to manage the managed care program. |
