Sensitivity of a Clinical Decision Rule and Early Computed Tomography in Aneurysmal Subarachnoid Hemorrhage

Introduction Application of a clinical decision rule for subarachnoid hemorrhage, in combination with cranial computed tomography (CT) performed within six hours of ictus (early cranial CT), may be able to reasonably exclude a diagnosis of aneurysmal subarachnoid hemorrhage (aSAH). This study’s objective was to examine the sensitivity of both early cranial CT and a previously validated clinical decision rule among emergency department (ED) patients with aSAH and a normal mental status. Methods Patients were evaluated in the 21 EDs of an integrated health delivery system between January 2007 and June 2013. We identified by chart review a retrospective cohort of patients diagnosed with aSAH in the setting of a normal mental status and performance of early cranial CT. Variables comprising the SAH clinical decision rule (age ≥40, presence of neck pain or stiffness, headache onset with exertion, loss of consciousness at headache onset) were abstracted from the chart and assessed for inter-rater reliability. Results One hundred fifty-five patients with aSAH met study inclusion criteria. The sensitivity of early cranial CT was 95.5% (95% CI [90.9–98.2]). The sensitivity of the SAH clinical decision rule was also 95.5% (95% CI [90.9–98.2]). Since all false negative cases for each diagnostic modality were mutually independent, the combined use of both early cranial CT and the clinical decision rule improved sensitivity to 100% (95% CI [97.6–100.0]). Conclusion Neither early cranial CT nor the SAH clinical decision rule demonstrated ideal sensitivity for aSAH in this retrospective cohort. However, the combination of both strategies might optimize sensitivity for this life-threatening disease.


INTRODUCTION Background
Approximately 80% of non-traumatic cases of subarachnoid hemorrhage are attributable to ruptured cerebral aneurysms, for which delays in definitive aneurysm treatment can increase the risk of disability or death. 1,2 While the vast majority of aneurysmal subarachnoid hemorrhage (aSAH) cases are identified by cranial computed tomography (CT), the sensitivity of CT diminishes with time such that lumbar Sensitivity of a Clinical Decision Rule for SAH Mark et al. (absence of all the following: age >40, neck pain or stiffness, headache onset with exertion, loss of consciousness at headache onset) in such a clinical scenario might further reduce the posterior probability of CT-negative SAH to an acceptable level of risk. [8][9][10] Goals of this investigation We sought to further examine the potential incremental gain in sensitivity when applying a previously validated SAH clinical decision rule to a cohort of patients diagnosed with aSAH after presenting to the ED with normal mental status and undergoing early cranial CT.

Study Population
We screened electronic health records of patients treated within the Kaiser Permanente Northern California (KPNC) integrated healthcare delivery system between January 2007 and June 2013 for case inclusion if they had an ED or hospital encounter with an associated International Statistical Classification of Diseases and Related Health Problems, ninth edition (ICD-9) diagnosis code of SAH (430). Emergency care within KPNC is provided through 21 non-academic medical center-based EDs, serving a population of over 3.3 million Kaiser Foundation Health Plan (KFHP) members. This study was part of a larger project examining outcomes following misdiagnosis of aSAH. Patients were electronically excluded if they had an ICD-9 coded diagnosis of head or neck trauma within 24 hours of the index encounter, lacked continuous KFHP membership within the two weeks preceding diagnosis, were under 18 years of age or had a prior diagnosis of SAH between 2002 and 2006. Data on age, sex and race were electronically collected. We then manually reviewed charts for the following inclusion criteria: initial diagnosis at a KPNC ED, Hunt-Hess clinical grade of 1 or 2 at the time of ED presentation, non-contrast cranial CT imaging within six hours of headache onset, either evidence of SAH on non-contrast cranial CT or greater than five red blood cells per microliter on cerebrospinal fluid analysis, and angiographic evidence of cerebral aneurysm thought to be consistent with the clinical presentation and pattern of hemorrhage visualized on imaging, if applicable. The study was approved by the Kaiser Foundation Research Institute Institutional Review Board with a waiver of the requirement for informed consent.

Methods and Measurements
Two investigators (DGM and MVK) conducted a structured explicit chart review and abstraction of records using a standardized paper form as part of a larger study examining outcomes following misdiagnosis of aSAH. 11 Abstractors confirmed the inclusion criteria and the final radiologist interpretation of the initial cranial CT, the location and size of the culprit aneurysm and documentation of the presence or absence of the following variables: neck pain or stiffness, loss of consciousness, physical exertion at the time of headache onset, need for external ventricular drainage and treatment of vasospasm during hospitalization. A best modified Rankin Scale (mRS) score at one year was assigned by reviewing neurosurgical, rehabilitation services and primary care clinical notes following hospital discharge, if applicable, using previously validated methodology. 12 We considered a mRS score ≤2 a favorable neurologic outcome. Both abstractors reviewed 20% of the sample to establish the inter-rater reliability of the following variables with an estimated error margin less than 20%: early cranial CT (inclusion criteria), Hunt-Hess grade at ED presentation and the clinical decision rule. 13 All CT examinations were performed without contrast using multi-slice cine technology (16 slice or higher). Either general radiologists or neuroradiologists made the final interpretation of CT images.

Outcomes and Analysis
The primary outcome of interest was the combined sensitivity of early cranial CT and the SAH clinical decision rule (a negative result for the latter being defined as absence of all four clinical criteria). Secondary outcomes were the independent sensitivities of early cranial CT and the SAH clinical decision rule. Missing variables from the SAH clinical decision rule were imputed as being absent to provide the most conservative estimate of sensitivity. We calculated binomial confidence intervals (CI) using the Clopper-Pearson (exact) method. All statistical analyses were performed using STATA v 13.0 (College Station, TX).

Characteristics of study subjects
We identified 155 patients following application of exclusion and inclusion criteria ( Figure). The median age was 55 years and 79% were female. Hunt-Hess grade was 2 in 95% of patients, though none of these had notation of a cranial nerve deficit upon initial ED evaluation. The most common aneurysm location was the anterior communicating artery (30%), followed by the posterior communicating artery (21%). Eighty percent of patients had a favorable neurologic outcome one year from initial hospitalization. Summary statistics of the study population are provided in Table 1.

Main results
Early cranial CT was reported as positive for SAH in 148 patients, yielding an estimated sensitivity of 95.5% (95% CI [90.9-98.2]). The SAH clinical decision rule was likewise positive in 148 patients with the same estimated sensitivity (95.5%, 95% CI [90.0-98.2]). Since the false negative cases for early cranial CT were mutually independent from the false negative cases by the SAH clinical decision rule, the combined estimated sensitivity for application of both early cranial CT and the SAH clinical decision rule was 100% (95% CI [97.6-100.0]). Seven patients (4.5%) underwent lumbar puncture, all  of whom had negative early cranial CT interpretations. Pertinent details for false negative cases of early cranial CT and the SAH decision rule are provided in Tables 2 and 3

DISCUSSION
The goal of this study was to help clinicians further refine and understand current testing strategies for aSAH, specifically by highlighting the potential gain in sensitivity obtained with the "post-hoc" application of a SAH clinical decision rule following early cranial CT. While we recognize that this was not the original derivation or validation setting of the SAH clinical decision rule, we feel our study demonstrates that such an approach can potentially help inform shared decision-making between clinicians and patients when faced with uncertainty over the absolute sensitivity of early cranial CT, especially in the face of nonlow pretest probability.
This testing strategy is similar in concept to performing a serum d-dimer assay to reliably exclude a lower extremity deep vein thrombosis in a patient with a high pre-test probability for disease but a negative lower extremity ultrasound examination; only the combined sensitivities of the two tests offer a low enough post-test probability to forgo further testing. 14 Additionally, although early cranial CT failed to detect aSAH in seven out of 155 cases (4.5%) in our retrospective cohort, this is only a point estimate and is specific to our practice setting. The findings of 100% sensitivity in the prospective Perry et al. 4 cohort and the retrospective Backes et al. 3 study rightfully prompt consideration of using early CT alone to rule out aSAH in those particular practice settings. Important contextual differences between our study and prior reports involve both spectrum bias and radiologist staffing practices. Perry et al. 4 enrolled patients with acute headaches reaching maximal intensity within one hour. Backes et al. 3 retrospectively identified patients presenting to a SAH referral center with a 50% incidence of SAH among patients undergoing early cranial CT (as opposed to 13% in the Perry et al. study). Thus it is possible that our study cohort includes patients with less severe presentations of aSAH who may be less likely to manifest positive CT findings on early cranial CT. Likewise, radiology staffing at a tertiary neurosurgical referral center as in Backes et al. 3 is not representative of the vast majority of EDs. While the Perry et al. 4 study setting was similar to ours in that radiographic studies were interpreted by a mix of neuroradiologists and general radiologists who routinely interpreted cranial CTs, several of the medical centers participating in that study had active radiology residency training programs with over-reading of studies by faculty in the daytime, making it difficult to extrapolate that level of scrutiny to practicing radiologists in a non-academic hospital setting. Of note, one early cranial CT in that series was initially misinterpreted as being negative by a radiology trainee, and was only retrospectively re-interpreted as positive when a magnetic resonance angiogram performed several days later revealed an aneurysm. This example highlights the potential for introducing   hindsight bias by using final written radiology reports as the gold standard for CT interpretations, an issue for all studies on this topic to date. Finally, it is notable that all of the patients with false negative cranial CT studies met the age criteria of the SAH clinical decision rule. It is thus possible that a post-imaging rule could be further refined for improved specificity and ease of applicability.

LIMITATIONS
Given the retrospective nature of the study, appropriate characterization of early cranial CT is only as accurate as the available documentation. While the inter-rater agreement for abstraction of this dichotomous variable was 100%, we cannot exclude errors in reporting the actual time of headache onset such that some patients may have been both included and excluded inappropriately. However, such errors in reporting can occur in prospective observational studies as well, and thus is a more general limitation of using historical factors as part of any decision rule.
The completeness of our case identification was also limited by the accuracy of the diagnostic coding from hospital and ED encounters. However, we searched databases specific for diagnostic codes assigned during treatment within KPNC as well as those used to track services billed for outside of KPNC, thus making it unlikely that we failed to capture cases of aSAH that were transferred to a non-KPNC hospital. Regardless, it seems improbable that we failed to capture enough cases of aSAH to appreciably alter our results; for example, to raise the sensitivity point estimate for detection of aSAH by early cranial CT to 99.0%, we would require an additional 545 cases of aSAH, all with positive early cranial CT findings (693/700=0.99).
Given that we conservatively imputed missing variables for the SAH decision rule as being absent, it is also possible that the SAH clinical decision rule may have performed better than reported if missing variables were in fact present. However, recent prospective and retrospective validations of the same SAH clinical decision rule also revealed suboptimal sensitivities at 98.5% and 96.6%, respectively. 8,15 Additionally, we cannot comment on the specificity of the SAH clinical decision rule when applied following early cranial CT, given the case-only cohort design of our study. It is possible that such an approach would not reduce overall testing rates from current practice, although individual clinicians may be differentially influenced given variable testing thresholds. 16 Finally, we did not include potentially missed cases of aSAH who may have undergone early cranial CT imaging with no confirmatory (LP) or subsequent testing, only to later present with evidence of aneurysm rupture. Inclusion of such cases in this cohort would bias towards a lower sensitivity of early cranial CT since it is impossible to be certain that evidence of SAH would have been otherwise detected (i.e., by lumbar puncture) at the initial evaluation.

CONCLUSION
In summary, we found the sensitivity of early cranial CT for aSAH to be 95.5% among patients presenting to non-academic EDs in an integrated healthcare system. Application of a SAH clinical decision rule in addition to early cranial CT improved sensitivity to 100% (95% CI [97.6-100.0]). Prospective decision rule refinement and validation of this approach is warranted.