| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||
1 Department of Health Policy and Management, Graduate School of Public Health, University of Pittsburgh, Pennsylvania.
2 RAND, Pittsburgh, Pennsylvania.
Address correspondence to Nicholas G. Castle, PhD, Assistant Professor, A649 Crabtree Hall, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA 15261. E-mail: castlen{at}pitt.edu
 |
A |
|---|
 Top Abstract MethodsResultsDiscussionReferences |
|---|
Methods. Using nationally representative data from the Online Survey, Certification, and Reporting system for 2000 (N = 17,072), we calculated predicted quality scores using risk-adjusted models based on aggregate resident variables generated by hierarchical linear regression models for each of the five quality indicators.
Results. We observed significant variation in both the actual and risk-adjusted quality measures. The average risk-adjusted physical restraint quality score ranged from 8.4% to 12.8%; the average risk-adjusted catheterization quality score ranged from 3.6% to 7.7%; the average risk-adjusted contractures quality score ranged from 19.0% to 31.6%; the average risk-adjusted pressure ulcer quality score ranged from 3.8% to 7.6%; and the average risk-adjusted psychoactive medication quality score ranged from 47.8% to 56.9%. Eleven states had quality measures better than the risk-adjusted expectation for at least four of the five measures, and eight states were worse than expected in at least four of the five.
Conclusions. This study provides evidence that there is variation in quality indicators across states. These differences exist even after risk adjustment. Our results may be important for state regulators trying to understand and improve quality.
It is well recognized that the quality of health care in the United States is very uneven (1). Nursing facilities are among the health care institutions most commonly cited for poor quality care. For example, a recent report identified 25% of nursing facilities as having serious quality problems that can either harm residents or place them at risk of death (2). However, we could not identify any studies documenting the variation in case–mix adjusted quality from state to state. Yet several studies (e.g., 3,4) have identified various state-level policies as important factors affecting the delivery and quality of nursing facility care.
|
METHODS |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
All the quality indicators reported in this study came from the OSCAR data, and are based on their prevalence in the resident population. Aggregate resident characteristics were included as risk adjustors in our development of predicted quality scores, and also came from the OSCAR.
Quality Indicators
The dependent variables were the facility-level rates of physical restraint use, urethral catheterization, contractures, pressure ulcers, and psychoactive medication use, used as quality indicators in this study. Selection of these indicators was based on their availability in the OSCAR data and previous use by researchers.
Use of vests, wrist restraints, ankle restraints, and/or geri-chairs is included as physical restraint. The 2000 OSCAR shows that nationally, 13% of nursing facility residents were physically restrained. The prevalence of physical restraint use is an important quality indicator, because the use of restraints is associated with an increased risk of morbidity and mortality in nursing facility residents (5). Lower levels of physical restraint use are generally regarded as beneficial. The 2000 OSCAR shows that, nationally, 10% of nursing facility residents had indwelling urethral catheters. Spector and Takada (6) found that residents in facilities that had moderate-to-high use of urethral catheterization had twice the probability of functional decline, compared to residents in low-use facilities. Contractures are an abnormal shortening and stiffening of muscle tissue that can decrease the range of motion at a joint. This decrease can produce a change in gait and decrease in walking velocity—which are major risk factors for falls—and may also limit mobility in daily life. The 2000 OSCAR shows that, nationally, 11% of nursing facility residents had contractures. Contractures are frequently used as indicators of care quality as they are effectively postponed and corrected by exercise programs, massage, and physical therapy (7). Pressure ulcers influence both the comfort and the medical outcomes of nursing facility residents with impaired mobility. The 2000 OSCAR shows that, nationally, 6% of nursing facility residents had pressure ulcers. We examined antianxiety, sedative/hypnotic, and antipsychotic psychoactive drugs. The 2000 OSCAR shows that, nationally, 36% of nursing facility residents were given these psychoactive drugs. The general concern with these drugs is that the rates of use may be excessive and/or clinically unjustified.
Risk Adjustment
In general, unadjusted rates of quality indicators are a function of both quality and risk-factors of the nursing facility population. With each of the five quality indicators as dependent variables we used separate analyses to calculate predicted facility scores using the aggregate resident characteristics as independent variables. This approach adjusts the outcomes for the resident risk factors of each nursing facility population. In addition, we adjusted for facility staffing and other facility characteristics that may be associated with quality indicators. The full set of variables used in the risk-adjustment models are shown in Table 1.
|
Analysis
We examined the correlations among all the variables to identify whether the data had any problems of collinearity. Most correlations between variables were small (therefore, the covariance matrix is not reported), and based on a threshold of.8, the variables showed no problems of collinearity (12). Values for regression-tolerance statistics (not reported) showed no problems of multicollinearity.
The unit of analysis was the individual facility, and, as noted, the dependent variable was the facility rate of each quality indicator. We used hierarchical linear models (HLM) to calculate state rates of quality indicators adjusted for resident, staffing, and facility risk factors. Separate models were estimated for each quality indicator. HLM provides empirical Bayes estimates, which are more efficient than simply aggregating to the state level (13). In addition, this approach explicitly takes into account the hierarchical nature of the data (i.e., facilities are nested within states), which without correction can lead to biased standard errors (13). Predicted values were computed for each state by subtracting the state-specific random effect from the observed rate. The state-specific random effect captures systematic differences between states, after adjusting for case–mix. The predicted value, therefore, reflects the expected rate for each state, given its observed risk profile. All analyses were conducted with SAS PROC MIXED (SAS Institute, Inc., Cary, NC). Model fit was assessed with the likelihood ratio test and the pseudo-R2, both of which are based on the comparison of the risk-adjustment model to a null model with only an intercept.
For ease of presentation, we also computed the ratio of the observed rate to the expected rate from the HLM for each dependent variable. Ratios greater than one indicate that the rate is higher than would be expected based on risk factors alone, and ratios less than one imply that the rate is lower than would be expected.
|
RESULTS |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
A more fine-grained conclusion regarding the appropriateness of the risk-adjustment models can be drawn from examining the change in the variance components at the facility and state level. First, using the unadjusted data, we computed the intraclass correlation, a measure of the proportion of total variation at the state level, for each quality indicator. The intraclass correlation for catheters was.04, for psychoactive medications.05, for pressure sores.07, for restraints use.14, and for contractures.17. These results imply that, although most of the variation in quality outcomes is at the facility level, for restraints and contractures there is a sizeable component that can be attributed to states. After estimating the risk-adjustment models, the variance at the state level decreased. Specifically, the variance explained at the state level in the use of restraints was 4.1%, catheters was 13%, contractures was 12.5%, pressure sores was 13.1%, and psychoactive medications was 19.9%. After risk adjustment, however, there still existed a statistically significant variance component at the state level, indicating that unexplained variation remains.
Table 2 shows the year 2000 nonrisk-adjusted (observed) values, predicted values, and the ratio of observed to predicted values for each state and for each quality indicator. Taking, for example, the average percentage of residents physically restrained, at the state level (nonrisk adjusted) the values vary from 2.3% (IA) to 23.1% (LA), the predicted mean values range from 8.4% (IA) to 12.8% (VA), and the ratio of observed and risk-adjusted values range from 0.27 (IA) to 2.13 (LA).
|
|
DISCUSSION |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
A pattern is also evident for the overall risk-adjusted quality indicator scores. When these scores are summed for all states the scores for physical restraint use and psychoactive medication use are better than expected, and the scores for catheter use, contractures, and pressure ulcers are worse than expected. This implies that although the state level variance for all of the quality indicators we investigated is large, overall facilities in all states seem to be providing relatively better-than-expected care for physical restraint use and psychoactive medication use.
This positive finding for physical restraint use and psychoactive medication use may reflect the considerable attention these quality outcomes have received in recent years. Consumer groups such as the National Citizens Coalition for Nursing Home Reform (NCCNHR), professional groups such as the American Nurses Association (ANA), and accrediting bodies such as JCAHO have been particularly successful in sensitizing the public and caregivers about the indiscriminate use of restraints and psychoactive medication in nursing facilities (8). The same emphasis on other indicators of care quality is long overdue.
States also vary in the number of better-than-expected quality scores and worse-than-expected quality scores. Clearly, it would be advantageous to determine why some states have a greater number of better-than-expected quality scores than others have. The goal of the present study was to determine the amount of variation that exists between states, after controlling for aggregate resident risk factors, facility staffing, and other facility characteristics. In future analyses, it may be productive to examine the association between state long-term care policies and quality.
Concerns about the quality of health care and public accountability have increased in recent years resulting in greater efforts to provide consumers with information about the quality of care in various settings. Quality concerns have been particularly pronounced in nursing facilities where incidences of abuse have received considerable attention in the media and where frail elderly persons are particularly vulnerable. However, efforts to provide consumers with information about the quality of care of nursing facilities are relatively recent (e.g., Nursing Home Compare released in November 2002). One potential use of the information we present would be as a state report card of nursing facility quality indicators.
Initiatives such as Nursing Home Compare give consumers a tool to choose between nursing facilities. This empowerment is commendable; however, it should not replace state and/or federal government quality oversight. As a report card, the information we present can help keep state governments accountable for nursing facility quality. The information could also be used by state governments to direct quality-improvement activities and surveyor activities. We may also see some cross-fertilization of successful initiatives in these areas across state boundaries.
Study Limitations
This study has several limitations related to data quality and our methodological approach. One of this study's strengths is that it uses a readily available and widely used data source that enables the profiling of quality indicators across states. These data are commonly used as a source for nursing facility and resident characteristics, and for analyses assessing quality of care. Moreover, they are collected by agents within each state. Despite the strengths of the OSCAR data, it should be recognized that these data have several limitations. For example, we could not find any evidence addressing the reliability of the resident characteristics reported in the OSCAR. These data are collected at the facility level and, as such, only represent aggregate information for each facility. Moreover, the data are collected relatively infrequently (i.e., approximately yearly).
In future analyses, the use of Minimum Data Set data may produce more robust results. These data are collected on each resident, are updated quarterly, and provide more resident detail than can be found in the OSCAR. Although the MDS data does also have limitations, it is not widely available, and for analyses such as ours, using data from all 50 states is expensive. Mor and colleagues (14) recently describe in detail the advantages and limitations of using these data.
It is worth noting, however, that both the OSCAR and MDS may be subject to measurement error when examining quality measures. Schnelle and colleagues (15) recently determined that the MDS restraint use quality indicator may be subject to inaccurate self-reports or variability in restraint use definition at the facility level. It is likely that the same measurement error exists in the OSCAR. Schnelle and colleagues (15) used a facility sample from one state, but given the substantial state influence on nursing facility reporting and recording activities, some systematic measurement error may exist among states. For our study, between-state differences in definitions of what constitutes a specific quality measure or differences in reporting for specific quality measures may be confounding, influencing the large variance among states we find on these indicators.
A further limitation of our approach is that the results may only be representative at the point of data collection. As with all cross-sectional analyses, our results do not account for possible fluctuations that could occur with the variables we use. In reporting results for individual facilities this may present some biases; however, our results are aggregated to the state level limiting this bias. Still, the use of more data points would improve our models. For example, we could examine states with improving or declining quality indicator scores. Wray and Hollingsworth (16) also discuss the susceptibility of reports such as ours to temporary confounders and errors resulting from short time frames.
The second issue important to the acceptance of the information provided is risk adjustment. Our quality indicators are proportional dependent variables. A substantial body of research using similar proportional dependent variables in the hospital setting exists. Thus, our approach has some precedent. However, other studies in the hospital arena (e.g., 17) have demonstrated that outcome results can be dependent on the method of risk adjustment and that our analyses may be similarly limited. Mukamel, Dick, and Spector (18) have also found differences in rank ordering when different definitions of quality indicators are used (e.g., the ratio of observed rates to expected rates or as the difference between observed and expected rates). In our case, in sensitivity analyses using the difference between observed and expected rates, the results were highly similar to those we present.
We are sensitive to these risk-adjustment issues, as the rankings we present are likely to be criticized based on the risk-adjustment models. There is considerable debate over the type of risk adjustment that should be used in long-term care and the factors that should be included in these models. Spector and Mukamel (19) discuss numerous alternative risk-adjustment techniques that could be used in analyses such as ours. Preliminary analyses using some of these techniques did not significantly change our results (results not shown). For example, analyses using independent variables lagged by 1 year and within state percentile rankings were not significantly different from those we present. Our results were also robust when outlier facilities for the quality indicators were excluded. But because we cannot discount the possibility that our results are dependent on the risk-adjustment method used, we also present summaries of the total number of better-than-expected quality scores for each state, which is a more conservative ranking of states. In sensitivity analyses, we were able to show that, with this conservative approach, the relative rankings for states were consistent—regardless of the risk-adjustment models used (not shown). It should also be noted that the MDS data described above could be used to create more robust risk adjustment, including the use of more fine-grained lagged measures. However, the fact remains that a gold standard to risk-adjust nursing facility quality indicators, even with the MDS, does not currently exist.
Conclusion
This study provides evidence that there is variation in nursing facility quality of care indicators across states. These differences exist even after risk adjustment. We believe that the state policy environment has the potential to influence the quality of nursing facility care, although our results show that this potential may vary across different quality indicators. Future analyses will focus on the aspects of state and federal policy that are most likely to influence quality of care across states.
|
F |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
Received March 3, 2004
Accepted July 14, 2004
|
References |
|---|
|
TopAbstractMethodsResultsDiscussionReferences |
|---|
| ||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
