Statistical Modeling of the Volume-Outcome Effect for Carotid Endarterectomy for 10 Years of a Statewide Database

Background: The carotid endarterectomy (CEA) remains one of the most common vascular surgical procedures in the United States: in 1996, 108,000 carotid endarterectomies were performed. More than 50 years after its implementation by Eastcott and Robb, CEA remains the gold standard treatment for carotid artery disease. Despite the proven efficacy of the CEA, questions remain as to whether all patients have access to the high quality surgeons and hospitals reflected in the North American Symptomatic Carotid Endarterectomy Trial (1991), Asymptomatic Carotid Atherosclerosis Study (1995), and other pivotal studies.
Studies of other surgically treated diseases support the notion that practice makes perfect: those surgeons or hospitals performing the greatest number of surgeries yearly provide the best outcomes. Numerous studies of CEA have established tiers of surgeons or hospitals performing low- middle- and high-numbers of CEAs per year and demonstrated differences in outcomes among these strata. We aimed to achieve accurate statistical modeling of a putative relationship between carotid endarterectomy (CEA) annual surgeon and hospital volume and in-hospital mortality.
We hypothesized that we could establish evidence-guided volume cutoffs for hospital and surgeon volume using standard statistical techniques. By allowing the data of 10 years of the Maryland hospital discharge database to drive the analysis, we hypothesized that we could find the best-supported volume categories for CEA. Furthermore, we hypothesized that a model including other covariates such as age, race, gender and comorbidities would establish the relative contribution of surgeon and hospital volume compared to these other factors.
Methods: The present study was a secondary data analysis of 10 years (1994-2003) of the Maryland Health Services Cost Review Commission (HSCRC) database. Established in 1971 by the Maryland legislature, this organization aims primarily to contain rising medical costs. In the process, the HSCRC collects extensive information on all patients and medical procedures occurring within the state of Maryland. We analyzed the HSCRC dataset over the abovementioned 10-year period, using a previously reported algorithm to identify 23,237 patients undergoing CEA. The following diagnoses were included: 1) International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) procedure code 38.12 (endarterectomy of the vessels of the head and neck other than intracranial vessels) in the primary coding position but not in any secondary position, or 2) Diagnosis code 433.00 to 433.91 (occlusion/stenosis, precerebral artery), or 3) the Diagnosis-Related Group (DRG) 5 (extracranial vascular procedure). The accuracy of this algorithm was confirmed through a previously published chart review. Patients receiving CEA concurrently with another procedure, such as coronary artery bypass grafting, were excluded from the analysis.
We analyzed the relationship between in-hospital death and annual surgeon or hospital volume. Annual surgeon volume was defined by the total number of procedures performed by a surgeon for their total time within the dataset divided by the number of years the surgeon was included in the dataset. Thus a surgeon who was included in the dataset for only five years would have an average volume that could be compared to any other surgeon who was included for any amount of time between one and ten years.
Crude odds ratios of death were first determined by logistic regression for annual surgeon volume and annual hospital volume. Heterogeneity by calendar year was explored by performing the analysis within each year. Non-linear relationships between death and average annual surgeon or hospital volume were explored by examining logit transformed lowess smoothing functions. Rough identification of spline knots from the plots was followed by using random effect models and inspecting likelihood of the data under each combination of spline knots. Consideration was given to other values around the knots initially identified, +/- 5 CEAs/year. A functional form for age was similarly determined by inspection of logit transformed lowess smoothed plots.
Analysis proceeded similarly for looking at hospital volumes. Annual average hospital volume was defined analogously as for annual average surgeon volume: number of procedures performed in the hospital in the complete 10-year dataset, divided by the number of years the hospital was included in the dataset.
Effect of patient co-morbidity was determined through use of the Charlson Index Deyo modification. Charlson Index values of 1 and 2 were compared a reference of >= 3 for their association with risk of death following CEA.A marginal model with generalized estimating equations (GEE) was used to represent population-average response as a function of covariates and to account for clustering in the data. Clustering was accounted for only at the surgeon level, not hospital.
All measures of statistical significance were based on alpha = 0.05. Stata SE, version 9.0, from Stata Corporation (College Station, TX) was used for data analysis. The final model of general estimating equation GEE) was created in SAS (SAS Institute Inc., Cary, NC) using unstructured correlation.
Results: From 1994-2003, CEA was performed on 23,237 patients in Maryland, in 47 hospitals by 438 surgeons. Of these patients, 465 were missing surgeon identifiers, leaving a dataset of 22,772 observations. This sample consisted of 54.7% men and 45.3% women, ranging in age from 33-99 years (mean, 70.6). The vast majority of the patients were white (21,229 or 91.4%); 1,682 were black (7.2%). There were 123 in-hospital deaths (0.54%) over this 10-year period among those with surgeon identifiers.
This distribution of CEAs among surgeons followed a roughly inverse power relationship, with the majority of surgeons performing an average of one CEA/year. The crude odds ratio of death for the entire surgeon dataset was 0.9838, meaning that the odds of death decreased by an average of 0.0162 for each additional annual procedure.
Examination of the logit transformed lowess smoothed functions suggested two knots for surgeon volume: around 5 CEAs/year and around 20 CEAs/year. Using random effects models we examined log likelihoods of different models around these knots (i.e. 3, 4, 5, and 15-25, etc.) and found the highest log likelihood to be rendered by knots at 3 CEAs/year and 15 CEAs/year. The lowess smoothed plot for age clearly suggested an inflection at 60 years, a value that was subsequently borne out by examination of ages grouped around 60 years.
Examination of the logit transformed smooth curve for hospital volumes suggested a single spline at annual hospital volume of 130 CEAs/year.
The significance of patient co-morbidity was reflected in the statistically-significant effect of Charlson Index scores of 1, 2 and >= 3, using score >= 3 as the reference group.
Logistic regression rendered the values for odds ratios with associated 95% confidence intervals and P values shown in Table I. As can be seen in the bolded values in the table, surgeon volume of 3-14 CEAs/year was highly significant with respect to odds of death. For each average increase in surgeon procedure per year, the odds of death decreased by 0.065 when controlling for hospital volume, age and comorbidity. Hospitals that saw >= 130 CEAs/year had an odds ratio of death of 0.945 per additional procedure, P = 0.0291, or 0.055 decrease in the odds of death. For each additional year in age for those younger than 60 years, the odds ratio per year of age was 0.936, but statistically non-significant (P = 0.0859). However the age relationship appeared U-shaped: patients >= 60 had a ratio odds of death of 1.058 per additional year of age when controlled for hospital and surgeon volume, P < 0.0001. Finally, the Charlson comorbidity scores were highly significant, with each additional comorbidity adding significantly to the odds of death after CEA. Those with a single comorbidity on the Charlson Index had an odds ratio of death of 0.187, P < 0.0001 compared to those with 3 or more comorbidities, while those with two comorbidities had an odds ratio of 0.362, P < 0.0001.
The inclusion of gender in the model did not change any of the inferences and was itself insignificant, with a P value of 0.7715. Female gender had an odds ratio of death of 0.948 compared to men when controlling for the covariates above. Non-white race (black and other) had a higher odds of death after CEA than whites when controlling for other factors, but this relationship was non-significant.
Conclusions: We have demonstrated a technique for the rigorous statistical analysis of volume-outcome data and have found a modest volume effect in this 10-year dataset from Maryland for CEA. Higher volume CEA surgeons had lower odds of death, particularly those performing 3-14 CEAs/year. Patients attending a surgeon performing an average of 15 CEAs annually should be assured that their surgeon is statistically equivalent to one performing any number higher than this, all other variables being equal. Hospital volume was not seen to be as significant a predictor of post-operative death in this study, with only high volume hospitals (>= 130 CEAs/year) showing a statistically significant decrease in the odds ratio of death. Patient comorbidity was a significant predictor of in-hospital post-operative death, confirming the need for careful patient selection for CEA.