Emergency Department Computed Tomography Use for Non-traumatic Abdominal Pain: Minimal Variability

Introduction Variability in the use of computed tomography (CT) between providers in the emergency department (ED) suggests that CT is ordered on a provider rather than a patient level. We aimed to evaluate the variability of CT ordering practices for non-traumatic abdominal pain (NTAP) across physicians in the ED using patient-visit and physician-level factors. Methods We conducted a retrospective study among 6,409 ED visits for NTAP from January 1 to December 31, 2012, at a large, urban, academic, tertiary-care hospital. We used a two-level hierarchical logistic regression model to estimate inter-physician variation. Intraclass correlation coefficient (ICC) was calculated. Results The hierarchical logistic regression analyses showed that patient-visit factors including younger age, arrival mode by ambulance, prior CT, >79 ED arrivals in the previous four hours, and ultrasound had statistically significant negative associations with physician CT ordering, while surgical team admission and white blood count (WBC) >12.5 K/millimeter cubed (mm3) had statistically significant positive associations with physician CT ordering. With physician-level factors, only physicians with >21 years experience after medical school graduation showed statistical significance negatively associated with physician CT ordering. Our data demonstrated increased CT ordering from the mean in only one out of 43 providers (2.3%), which indicated limited variation across physicians to order CT. After adjusting for patient-visit and physician-level factors, the calculated ICC was 1.46%. Conclusion We found minimal physician variability in CT ordering practices for NTAP. Patient-visit factors such as age, arrival mode, admission team, prior CT, ED arrivals in previous four hours, ultrasound, and WBC count were found to largely influence CT ordering practices.


INTRODUCTION
Computed tomography (CT) utilization in the emergency department (ED) has increased significantly in the past 30 years. 1 A 330% rise was observed from 1996 to 2007 in a retrospective study of the National Hospital Ambulatory Medical Care Survey, with utilization for non-traumatic

Population Health Research Capsule
What do we already know about this issue? Studies examining computed tomography (CT) use among emergency physicians for overall use and non-traumatic abdominal pain (NTAP) have demonstrated minimal and significant variability, respectively.
What was the research question? This study evaluated the variability of CT ordering practices for NTAP among emergency physicians.
What was the major finding of the study? The use of CT by emergency physicians for NTAP showed minimal variability and was influenced by patient-visit factors.
How does this improve population health? Findings contribute to evidence to further clarify CT appropriate use to optimize resource utilization.
Studies measuring CT use and associated outcomes and ordering practices for NTAP have not been in agreement. Rates of change in diagnosis and change in disposition for NTAP in five studies have been as high as 54% and 40%, respectively. [5][6][7][8][9] Yet three studies describe an increase in diagnostic specificity for NTAP but with no change in admission rates, missed surgical diagnoses, or six-month mortality. [10][11][12] One study showed minimal variability in physician ordering practices when examining overall CT use, while another showed significant variability when examining exclusively NTAP CT use. [13][14][15] Increased CT use adds additional costs to clinical evaluation and treatment. Furthermore, concerns related to radiation exposure and the risks of benign, incidental findings are legitimate. 16,17 Within this context of equivocal risk-benefit and cost-benefit understanding, examination of variability in CT ordering practices across physicians, as well as against physician level and patient-visit level predictive factors will contribute to the identification of appropriate use [18][19][20] and may suggest guideline modifications that could result in decreased imaging with similar or improved outcomes. We examined the variability of CT ordering practices for NTAP across physicians in the ED using both patient-visit and physicianlevel factors. This focus adds to previously published literature, which has predominantly examined physician-level factors only or overall CT use, respectively.

Study Design and Data Collection
We conducted a retrospective cohort design, reviewing the electronic medical records (EMR) of patients visiting the ED at a large, urban, academic, tertiary-care hospital. EMR patient visit-level data included demographics, dates and times of ED registration, discharge and admission, diagnosis, attending physician, dates and times of image order, test name and results. The physician's gender and education background was extracted from the public-access hospital website. This study was approved by the institutional review board with informed consent waiver and was compliant with the Health Insurance Portability and Accountability Act (HIPAA).
We included all patient visits from January 1 to December 31, 2012, with a chief complaint of abdominal pain. We excluded patient visits from the trauma unit as well as those with pregnancy, patients less than 18 years old, with attending physician's annual NTAP visits < 50 (similar to Levine et al.), 15 with incomplete radiology data, without attending physician, or any visit associated with trauma. See Figure 1 for the detailed exclusions.

Outcome Measure and Predictor Variables
The primary outcome was whether a physician ordered a CT during a patient's ED visit for NTAP. We investigated both patient-visit and physician-level factors as predictor variables. Patient-visit factors included patient gender; age; arrival mode (walk-in, ambulance, or indeterminate); acuity (determined using Emergency Severity Index [ESI] -most severe, more severe, severe, less severe, or least severe); arrival time (weekday vs. weekend, and by shift -day, evening, or night); disposition (discharge, admit, observation, against medical advice/absent without leave/ left without being seen, or indeterminate); admission team (surgical team, non-surgical team, or not admitted);whether or not the patient had a prior CT abdomen/pelvis; current ED volume (evaluated by counting the number of ED arrivals in the previous four hours); whether or not there was use of diagnostic ultrasound; first white blood count (WBC) count; first hemoglobin count; and first hematocrit count. Physician-level predictor variables included gender, years since completing medical school, whether or not a physician completed a fellowship, whether or not there was involvement of advanced triage (a provider with ability to initiate orders prior to full evaluation), and annual ED visit volume (sum of patient visits supervised by each physician throughout 2012). The numeric variables (i.e., ED arrivals in previous four hours, WBC count, hemoglobin count, hematocrit count, and physician's annual ED visit volume) were all categorized into quartiles. 13

Data Analysis
We conducted preliminary analyses to summarize patientvisit and physician-level characteristics by CT ordering status. Univariate and multivariate generalized linear models with repeated measures were performed to investigate the associations of patient-visit and physician-level factors, respectively. We applied the iterative fitting algorithm for repeated measures in modeling to avoid the violation of the assumption of independence due to the multiple patient visits cared for by the same physician. We used a two-level hierarchical logistic regression model with physician-specific random intercepts developed by Dr. Sistrom 13 to study the association of CT ordering with patient-visit and physicianlevel factors. The estimated physician-specific intercepts and associated standard errors were transformed by exponentiation to get the adjusted odds ratios with 95% confidence intervals (CI) for each physician.
To estimate the proportion of total variation attributable to the physician level after adjusting for the patient-visit and physician covariates, we calculated the intraclass correlation coefficient (ICC) by using the estimated variance of the physician-specific intercepts from the two-level hierarchical logistic regression model and an estimate of the standard logistic function variance of π 2 /3. We also calculated a reliability estimate for each physician using the formula, OIV/ (OIV+SEPI) 2 , where OIV is the overall intercept variance, SEPI is the standard error for each physician, and both are produced directly from the multilevel model. The aggregate reliability with 95% CI was produced by averaging the reliability estimate for each physician. We performed all analyses using SAS v.9.4 (SAS Institute, Carry, NC, USA), and statistical significance was evaluated at the 0.05 level.

RESULTS
Of 95,153 total ED patient visits from January 1 to December 31, 2012, 8,222 visits were for NTAP by chief complaint. After the exclusions of 418 visits with pregnancy by chief complaint and 468 by positive beta-human chorionic gonadotropin (ß-hCG), 56 visits from patients less than 18 years old, 232 visits with incomplete radiology data, 457 visits without an attending physician, 19 visits associated with trauma . , and another 163 visits supervised by seven providers with less than 50 annual visits, the final study population comprised 6,409 patient visits. Figure 1 shows the flow chart of sampling in detail. In addition, over one third of patient visits without CT did not have the lab/record of a WBC count, hematocrit, and hemoglobin, while over 96% among the patient visits with CT ordering had these records. During the study period, 43 physicians saw the sampled ED visits. Table 2 shows the characteristics of these physicians. Over 50% of them had 10 years or longer experience after completing medical school. Over 70% of the physicians did not complete a fellowship. These physicians provided care with the median annual NTAP visit volume of 138 (interquartile range [IQR]: 97−209), and median CT ordering rate of 27.1% (IQR: 22.9−30.5%). Table 3 shows the unadjusted and adjusted odds ratios of CT ordering for the patient-visit variables. The univariate analyses showed that CT ordering was statistically significantly higher in the patients who were male, older, with severe or higher acuity, admitted by surgical team, had a WBC count >12.5 K/mm 3 , hematocrit count >45%, and hemoglobin count >17.1 g/dL. In the multivariate model, compared to the patients aged 44-63 years old, the odds of CT imaging for younger patients significantly decreased 16-36%, but increased over 35% for older patients; the patients who arrived by ambulance (vs. walk-in) (odds ratio [OR] [0.75]; 95% CI  In the final multilevel model, we included all patient-visit and physician-level factors together with physician-specific random effect. Table 5 shows the results of each of the patient-visit and physician-level variables; Table 6 shows only those variables that were statistically significant. The patientvisit variables showed similar associations as those in the multivariate analysis above, whereas among physician-level variables, only physicians who had >21 years of experience after graduation from medical school showed statistical significance and these physicians were less likely to order CT (OR [0.68]; 95% CI [0.48-0.96]; P = 0.03) compared to those with 10-21 years experience. Figure 2A shows the observed and predicted CT ordering rates for individual physicians plotted in ascending observed order. The predicted CT ordering rates accounted for fixed patient-visit and physician-level variables, but not for the random physician-specific intercepts. Figure 2B shows the corresponding physician-specific odds with 95% CIs for CT ordering. ORs less than one indicated the physician was less likely to order a CT; and ORs greater than one indicated higher tendency. There was only one out of 43 physicians  (2.3%) with the 95% CI of OR not intersecting one, which indicated limited variation across physicians to order CT. In the reduced model including physician-specific random intercept only, the calculated ICC was 4.73%. After adding the patient-visit and physician-level variables, the ICC was reduced to 1.46%. The estimate of reliability of the physicianspecific intercepts was 0.62 (95% CI [0.61-0.64]).

DISCUSSION
Our study found minimal physician variability in CT utilization. Moreover, numerous patient-visit factors were statistically significantly associated with CT use. While the identification of patient factors related to CT utilization is not new, our study adds to previous literature by demonstrating the overwhelming magnitude that patient-visit factors (and the minimal role that physician factors) contribute to CT ordering variability within the context of NTAP.
Both the calculated ICC and estimated reliability in our study suggested minimal physician variability in CT ordering practice, which was in accordance with the results reported by Wong et al. 13 Specifically, "for provider profiling purposes, when reliability is above 70%, meaningful difference between some physicians (called 'outliers') and the mean are discernible; at 90% reliability, difference between pairs of physicians are meaningful." 13 Therefore, considering that our reliability was below 70%, no meaningful difference between physicians was discernible in our study. Specifically, the ICC in this study represents the percent of variability in CT ordering that could be attributed to a particular physician.
Thus, given the ICC was reduced from 4.7% to 1.46% after controlling for patient-visit factors and physician factors, two points should be highlighted. First, consideration should be given to controlling for patient-visit factors when examining resource utilization. Second, given that physicians contribute ostensibly only 1.46% to total CT use variability, care should be used when identifying outliers for overuse or underuse. Our data demonstrated increased CT ordering from the mean in one out of 43 providers. That being said, we have   not overstated the provider's difference in utilization given the minimal physician influence over CT use found in this study. When examining physician factors separately we found years after completing medical school, fellowship, and advanced triage physician to be statistically significantly negatively associated with CT ordering. However, in the fixedeffects model considering physician and patient-visit factors jointly, only the subset of physicians with the longest period of time from completing medical school was statistically significantly negatively associated with imaging ordering, while patient-visit factors were shown to have a larger magnitude of association over CT imaging-ordering practices.
Some studies have shown that physician factors have minimal predictive value on ordering practices, 4,13 which were in accordance with our results. After considering all patientvisit and physician-level factors in our multilevel analyses, most physician factors were not statistically significantly associated with CT ordering. Notwithstanding, our findings contrast with studies that have shown physician age, board certification, and risk-tolerance to have statistical significance with respect to CT ordering. 15,[21][22][23][24] Differences in population, sampling, predictors considered, and/or the sample source may explain discordance among these studies. For example, shared decision-making in academic settings may serve to dampen image-ordering provider variability, and chief complaints such as trauma or head injury may carry unique considerations related to mechanism when compared to NTAP. 21 Conversely, elderly patient visits are associated with increased CT use due to their increased risk for abdominal pathology and their lessreliable physical exams. 25 We found that older patients were more likely to have CT as a part of their work-up. This is consistent with the benefits Volume 19, no. 5: September 2018  of CT in diagnosing the source for NTAP in the elderly, whose clinical presentation is a diagnostic challenge. 11 For the elderly, the etiology of NTAP often presents atypically, and abdominal tenderness or lack thereof may not be representative of the underlying pathology. 5,26 A prior CT was negatively associated with CT ordering in our study. Ostensibly, if a patient was already known to have an abdominal pathology, they may have been managed under the assumption of an acute flare of this condition, which did not require repeat imaging, in so far as their presentation is not overtly suggestive of severe progression. For example, a patient with a recently diagnosed renal or ureteral stone on CT would be unlikely to have a repeat scan as it has been shown that repeat CT in this setting does not provide additional benefit but potentially increases risk. 27,28 Moreover, if a recent CT is available, this may influence the provider to weigh concerns of radiation exposure against possible minimal added-benefit from repeat imaging in a patient with a previously negative scan or with chronic abdominal disease (e.g., a patient with inflammatory bowel disease may not receive a CT if they have recently had imaging). [29][30][31] In our study, a radiology ultrasound performed during the patient-visit was negatively associated with CT ordering. This is consistent with previous studies, which demonstrated the ability of ultrasound to rule in or rule out pathology. 20,32 While our study did not explicitly examine other imaging modalities, ultrasound potentially could make CT unnecessary in the setting of acute appendicitis or cholelithiasis. 33, 34 We did not evaluate emergency physician-performed bedside ultrasound. However, a bedside ultrasound that is clearly positive for cholecystitis could obviate the need for a CT. 35 Moreover, bedside ultrasound in the setting of renal colic could similarly influence CT use. 36 Using ED arrivals in the previous four hours as a surrogate for ED "busyness" or crowding, we found a busier ED negatively associated with CT imaging, which was different from the findings by Wong et al. 13 This may have been due to the time required to perform a CT and obtain results. Moreover, during high-volume periods in the ED, prioritization of CT use may have taken place (consciously or unconsciously) and disposition decisions may have been based more on clinical presentation. The varying effect of ED volume and crowding has been investigated, [37][38][39][40] and so impact on imaging ordering stands to reason.
Elevated WBC count was positively associated with ordering of CT. This further demonstrates the notion that patient severity would drive CT imaging. However, lack of significance of acuity represented by the ESI, while an imperfect metric, 41 makes this picture less clear. Moreover, sensitivity and specificity of WBC counts have unclear clinical significance in isolation so clinical decision scores such as the Alvarado score and the pediatric appendicitis score take into account multiple predictors. 42,43 It should be noted that our analysis of WBC count did not examine whether the WBC count resulted before or after a CT was ordered or deferred.    It bears mentioning that the presence of an advanced triage physician did not show statistical significance. Thus, whether order sets were initiated at triage or by the physician providing direct care to the patient did not impact CT utilization. Moreover, as in other studies 13,15 we did not evaluate the presence or absence of registered nurse-initiated order sets nor the possibility of resident CT ordering prior to attending consultation. Admission to a surgical team was positively associated with CT imaging. This finding suggests that patients admitted to surgery are surgical candidates and, therefore, likely to have more severe pathology. Thus, CT imaging may be used to confirm this acuity and contribute to surgical planning. 5,44,45 Arrival mode via ambulance was negatively associated with CT imaging. This was different from Wong et al. 13 who found that arrival via ambulance was positively associated with CT imaging. Moreover, the 2010 National Hospital Ambulatory Medical Care Survey demonstrated that 73% of ambulance ED visits are for patients > 65 years old, 46 an age group where increased use of CT was expected. While our arrival mode findings seemed contrary to that of severity driving CT imaging, one hypothesis could be that patients arriving by ambulance may have represented a disproportionate number of repeat visitors and may have had a recent CT in their medical records, which in our study was a negative predictor for CT use.
Our sample of 6,409 ED visits for NTAP was extracted from 95,153 ED visits. This is comparable to Wong et al. 13 who examined 88,851 ED visits for all types of imaging but did not provide subgroups by complaint. The subgroup of abdominal pain for Levine et al. 15 included 18, 614 ED visits for abdominal pain, and while this robust study was three times the size of our sample, they did not account for a number of statistically significant, patient-visit factors such as prior CT, prior ultrasound, surgical admitting team, WBC count, arrival mode, and ED volume. Thus, while our study sample was smaller by comparison, our examination and identification of strongly predictive patient-visit factors adds value to current evidence.