This article is mentioned in:

Introduction

The prevalence of cerebral aneurysm in the general population is estimated to be 1–5% (1). Ruptured intracranial aneurysms are the most important cause of non-traumatic subarachnoid hemorrhage, which is a medical emergency and can result in severe disability or mortality (2). Thus, the prompt diagnosis and treatment of intracranial aneurysm are of considerable importance for the outcome of the patient.

Conventional digital subtraction angiography (DSA) has been considered the gold standard for the detection and characterization of intracranial aneurysms due to its high spatial resolution and large field of view (3,4); however, it also has several disadvantages, including the relatively high cost and the high skill level required to perform the procedure. Furthermore, DSA is an invasive procedure associated with a small but definite risk of neurological morbidity (5). There is a requirement, therefore, to find an accurate, minimally invasive imaging method that is free from these complications. Computed tomography angiography (CTA), as a relatively non-invasive imaging method, has been widely used in the screening of patients with suspected intracranial aneurysms (6). As a result of the rapid improvement in multi-detector CTA technology, the diagnostic performance of multi-detector CTA for the detection of intracranial aneurysms is now approaching that of DSA (7); however, it exhibits a disadvantage in the detection of small-sized aneurysms that are located near the skull base due to the influence of overprojecting bone structures (8).

As CT technology has evolved and various subtraction and post-processing techniques have been developed, subtraction CTA, allowing bone-free visualization of aneurysms, has become possible for the diagnosis of intracranial aneurysms. There have been several reports on the potential usefulness of subtraction CTA in evaluating intracranial aneurysms; however, the results of these studies have been varied (6–8). The purpose of this meta-analysis was to calculate the sensitivity and specificity of subtraction CTA for the detection of cerebral aneurysms, in comparison with the reference standard of DSA.

Materials and methods

Search strategy

A systematic literature search up to January 1, 2013 was conducted in PubMed to identify any relevant studies. The search terms included ‘tomography, X-ray computed’ or ‘computed tomography angiography’, combined with ‘intracranial aneurysm’ or ‘subarachnoid hemorrhage’. Studies that were evidently irrelevant, based on a scan of the titles and abstracts, were excluded, while the remaining articles were assessed for relevance to the topic of interest by reading the full text. Furthermore, a manual search was conducted by checking the references of retrieved articles to find any additional published studies. All searches were conducted independently by 2 authors, prior to the results being compared. Any questions or discrepancies were resolved through discussion and consensus.

Study selection

To be eligible for inclusion in the meta-analysis, the studies had to satisfy 8 inclusion criteria: i) The patients were clinically suspected of having an intracranial aneurysm; ii) the diagnostic index test was bone subtraction CTA; iii) the reference standard was DSA or its combination with neurosurgical findings; iv) the condition of interest was the presence or absence of an intracranial aneurysm; v) 2×2 contingency tables were reconstructed on a per-patient or per-aneurysm basis; vi) the study had no limitations with regard to specific aneurysm types or locations; vii) the study included ≥20 patients, due to the increased likelihood of smaller studies suffering from selection bias; and viii) the study included ≥5 patients with and 5 patients without an aneurysm, so that the study provided meaningful numbers for sensitivity and specificity.

Data extraction

The study data were independently extracted by 2 researchers, and any disagreements were resolved through discussion and consensus. The following data were collected: Surname of the first author, year of study publication, country in which the study was performed, study design, age range of the study participants, index test and reference standard. The 2×2 count data were extracted on a per-patient basis and, if reported, on a per-aneurysm basis.

Assessment of study quality

Study quality was assessed independently by 2 researchers using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool, which includes 14 quality items (9); disagreement was resolved by consensus. This evidence-based tool was developed to assess the quality of primary studies of diagnostic accuracy. The QUADAS item 4 was scored positive if the delay between the index and reference tests was ≤3 days in all patients. For each study, a quality score was accumulated by assigning 1 point for each QUADAS item if fulfilled, 0.5 if unclear and 0 if not fulfilled. A score between 11 and 14 points was considered high quality and a score <11 points as low quality. A more detailed description of each item, together with a guide on how to score each item, is provided in the study by Whiting et al (9).

Statistical analysis

For all studies included in the meta-analysis, the individual sensitivities and specificities were recalculated from the 2×2 count data on a per-patient or per-aneurysm basis. Pooled summary estimates were obtained from a bivariate, mixed-effects, binary regression modeling framework. Model specification, estimation and prediction were performed with Stata software, version 11.0 (Stata Corp, College Station, TX, USA). A forest plot was generated that contained the individual study sensitivities and specificities with 95% confidence intervals (CIs) and the pooled sensitivity and specificity estimates. The areas under the receiver operating characteristic (ROC) curves (AUCs) were used to analyze the diagnostic precision.

The I2 statistic was used to examine whether the results of studies were homogeneous (10). This statistic uses the conventional Q statistic to calculate the percentage of tool variation heterogeneity on a scale ranging from 0% (no heterogeneity) to 100% (all variance due to heterogeneity). In contrast to the Q statistic, the I2 is less dependent on the number of studies included in a meta-analysis. I2 >50% suggested notable heterogeneity. When statistical heterogeneity was detected, sensitivity analyses were also performed.

In studies assessing test accuracy, one of the primary causes of heterogeneity is the threshold effect, which arises due to the use of different cut-offs or thresholds in the analyzed studies to define a positive (or negative) test result. The Spearman correlation coefficient between the logit of sensitivity and the logit of 1-specificity was calculated using Meta-Disc version 1.4, in order to determine the threshold effect (11). A threshold effect was indicated by the presence of a strong, positive correlation (P<0.05) (11).

The presence of publication bias was visually assessed through the production of a Deeks' funnel plot and an asymmetry test (12). In the Stata software, linear regression of log odds ratios on the inverse root of effective sample sizes was performed as a test for funnel plot asymmetry. A P-value of <0.10 was considered to be representative of statistically significant publication bias, suggesting that only the small studies that reported a high accuracy for subtraction CTA had been published, whereas the small studies that reported a lower accuracy had not been published. Data were analyzed with Meta-Disc version 1.4 and Stata version 11.0 software.

Results

Literature search

The initial search strategy retrieved a total of 5,224 citations. Following the screening of the titles and abstracts, 5,209 sources were excluded. The full texts of the remaining 15 sources were evaluated. Of these, 7 sources were excluded for reasons given in Fig. 1 (13–19). The remaining 8 studies were included (20–27).

Figure 1. Flow diagram of study identification. ICA, internal carotid artery.

The study characteristics are shown in Table I. The quality assessment scores ranged from 10.5 to 13.5, with a median study quality score of 12.5. Four studies were prospective, 2 studies were retrospective, and in 2 studies this was unclear. The 8 studies included 982 patients. Optional count data on a per-aneurysm basis in addition to count data on a per-patient basis were provided by all 8 studies (Table I).

Table I. Studies included in the meta-analysis.

Table I.

Studies included in the meta-analysis.

First author (ref.)	Year	Country	Design	No. of patients	Age, years	QUADAS score	No. of detector rows	RS	No. of aneurysms
Jayaraman (20)	2004	US	P	35	54	13.5	Multi-detector	DSA	26
Yoon (21)	2007	Korea	P	85	49.6	13.5	16	DSA	93
Romijn (22)	2008	Netherlands	NR	108	56	12.0	4	DSA	117
Li (23)	2009	China	P	76	48.0	12.5	64	DSA	76
Zhang (24)	2010	China	P	61	52.0	11.5	Dual-source	DSA	47
Ramasundara (25)	2010	Australia	R	36	NA	10.5	16/64	DSA	34
Luo (26)	2012	China	NR	56	48.0	12.5	320	DSA	51
Lu (27)	2012	China	R	525	52.0	13.5	Dual-source	3D DSA	459

[i] NR, not reported; P, prospective; R, retrospective; QUADAS, Quality Assessment of Diagnostic Accuracy Studies; DSA, digital subtraction angiography; 3D, three-dimensional; RS, reference standard. Data are presented as the mean.

Assessment of publication bias

On a per-patient and per-aneurysm basis, the funnel plot and regression test showed no significant publication bias (P=0.30 and P=0.53) (Fig. 2). This suggested that there were no smaller studies with lower diagnostic accuracies that had not been published.

Figure 2. Deeks funnel plot asymmetry test (P=0.30) showing log odds ratios on inverse root of ESS for the visualization of publication bias. ESS, effective sample size.

Analysis of heterogeneity and pooled sensitivity and specificity on a per-patient basis

The sensitivities ranged from 86 to 100% (Table II). Concerning sensitivity, the selected studies showed moderate heterogeneity (I2=81.2%). For specificity, low heterogeneity was observed (I2=36.0%); the specificity ranged from 89 to 100%. The overall pooled sensitivity was 99% (95% CI, 95–100%), and the pooled specificity was 94% (95% CI, 86–97%) (Fig. 3). The AUC was 0.99 (95% CI, 0.97–0.99) (Fig. 4).

Figure 3. Forest plot shows sensitivity and specificity from individual studies and pooled estimates. Dotted squares indicate mean sensitivity or specificity for each study; horizontal lines indicate the 95% CIs of sensitivity or specificity for each study; vertical, red, dashed lines indicate pooled sensitivity or specificity for all 8 studies. CI, confidence interval.

Figure 4. SROC curve with prediction and confidence contours. SROC, summary receiver operating characteristic; AUC, area under the curve; SENS, sensitivity; SPEC, specificity.

Table II. Count data, sensitivities and specificities of the included studies.

Table II.

Count data, sensitivities and specificities of the included studies.

	Per-patient							Per-aneurysm
First author (ref.)	Year	TP, n	FP, n	FN, n	TNa, n	Sensitivity, %a	Specificity, %a	TP, n	FP, n	FN, n	TNb, n	Sensitivity, %	Specificity, %
Jayaraman (20)	2004	18	1	3	13	85.7 (63.7–97.0)	92.9 (66.1–99.8)	20	1	6	13	76.9 (56.4–91.0)	92.9 (66.1–99.8)
Yoon (21)	2007	71	0	0	14	100.0 (94.9–100.0)	100.0 (76.8–100.0)	86	1	7	14	92.5 (85.1–96.9)	93.3 (68.1–99.8)
Romijn (22)	2008	87	2	1	18	98.9 (93.8–100.0)	90.0 (68.3–98.8)	106	3	11	18	90.6 (83.8–95.2)	85.7 (63.7–97.0)
Li (23)	2009	64	0	0	12	100.0 (94.4–100.0)	100.0 (73.5–100.0)	75	0	0	12	100.0 (95.2–100.0)	100.0 (73.5–100.0)
Zhang (24)	2010	41	1	0	19	100.0 (91.4–100.0)	95.0 (75.1–99.9)	45	1	2	19	95.7 (85.5–99.5)	95.0 (75.1–99.9)
Ramasundara (25)	2010	26	1	1	9	96.3 (81.0–99.9)	90.0 (55.5–99.7)	33	1	1	9	97.1 (84.7–99.9)	90.0 (55.5–99.7)
Luo (26)	2012	42	0	0	14	100.0 (91.6–100.0)	100.0 (76.8–100.0)	51	0	0	14	100.0 (93.0–100.0)	100.0 (76.8–100.0)
Lu (27)	2012	398	12	9	94	97.8 (95.8–99.0)	88.7 (81.1–94.0)	443	13	16	94	96.5 (94.4–98.0)	87.9 (80.1–93.4)

{ label (or @symbol) needed for fn[@id='tfn2-etm-0-0-3166'] } This table gives the 2×2 count data and the sensitivities and specificities of the 8 included studies on a per-patient and per-aneurysm basis.

a Mean (95% confdence interval)

b TN aneurysms were set equal to the TN patients. TP, true-positive; FP, false-positive; FN, false-negative; TN, true-negative.

A Spearman rank correlation was performed as a further test for the threshold effect and was determined to be 0.558 (P=0.151), which indicated that there was an absence of a notable threshold effect in the accuracy estimates among the individual studies.

The results of meta-regression indicated that the year the study was published, study design, quality score and blinding method were strongly associated with sensitivity, but not with specificity (Fig. 5).

Figure 5. Univariable meta-regression and subgroup analyses on a per-patient basis. CI, confidence interval.

Analysis of heterogeneity and pooled sensitivity and specificity on a per-aneurysm basis

The sensitivities ranged from 77 to 100% (Table II). Concerning sensitivity, the selected studies showed moderate heterogeneity (I2=84.3%). For specificity, low heterogeneity was observed (I2=6.9%); the specificity ranged from 86 to 100%. The overall pooled sensitivity was 96% (95% CI, 90–99%), and the pooled specificity was 91% (95% CI, 85–95%). The AUC was 0.96 (95% CI, 0.94–0.97).

A Spearman rank correlation was performed as a further test for the threshold effect and was determined to be 0.548 (P=0.160), which indicated that there was an absence of a notable threshold effect in the accuracy estimates among the individual studies.

False-negative CTA results

Forty-three intracranial aneurysms were missed at subtraction CTA. The location of the false-negative aneurysm was specified for 37 aneurysms (Table III). The size of the false-negative aneurysms was also provided in 37 cases: 19 were <3 mm, 15 were <5 mm and 3 were 5–10 mm in diameter. At least 22 of the missed aneurysms could be detected retrospectively.

Table III. Location of false-negative intracranial aneurysms.

Table III.

Location of false-negative intracranial aneurysms.

Location	No. of aneurysms
Anterior circulation
Pericallosal artery/ophthalmic artery	2
Anterior communicating artery/anterior cerebral artery	5
Internal carotid artery	11
Posterior communicating artery	5
Middle cerebral artery	9
Posterior circulation
Posterior cerebral artery	1
Posterior inferior cerebellar artery	2
Anterior superior cerebellar artery	1
Vertebral and basilar artery	1

Discussion

To the best of our knowledge, this study is the first meta-analysis of the diagnostic performance of subtraction CTA to detect cerebral aneurysms. The results show that subtraction CTA has a high diagnostic value for the detection of intracranial aneurysms. According to this meta-analysis of 8 studies, subtraction CTA has an overall sensitivity of ~99% (95% CI, 95–100%) and a high specificity of ~94% (95% CI, 86–97%) for diagnosing cerebral aneurysms on a per-patient basis. On a per-aneurysm basis, the diagnostic accuracy was only slightly lower.

Among the studies included in this meta-analysis, a total of 22 false-negative aneurysms at CTA could be identified retrospectively (20,21,23,24). These false-negative interpretations can therefore be categorized as perceptual in nature and could have been substantially bypassed by double reading.

The I2 statistic for sensitivity indicated the presence of notable heterogeneity, a finding that is consistent with previous meta-analyses investigating CTA and cerebral aneurysms (7,28). The Spearman correlation coefficient on a per-patient basis was 0.558 (P=0.151), which suggested the absence of a significant threshold effect. To determine whether other sources of heterogeneity existed, in addition to the threshold effect, a subgroup analysis was conducted to identify factors affecting heterogeneity. The results of the meta-regression showed that the year the study was published, study design, quality score and blinding method had a strong association with sensitivity.

Non-subtraction multi-detector CTA has a relatively high sensitivity and specificity for the detection of cerebral aneurysms (6–8); however, the detection of aneurysms adjacent to bone remains a challenging issue due to overlying bone structures. To potentially circumvent this limitation, a number of bone removal techniques, including subtraction and manual or automated bone editing, have been developed; however, these methods are associated with several disadvantages, such as the complexity of use, dependence on the user and the high dose of radiation. Manual bone editing in CTA is a time-consuming and user-dependent process that relies on a knowledge of vascular anatomy (13,14). Matched mask bone elimination (MMBE) is a relatively new technique that enables bone removal in an automatic and user-independent way (18,22). In CTA-MMBE, a second, non-enhanced scan is necessary for the identification of bony structures that could be masked in the CTA scan. The two consecutive volumetric scans expose the patient to more radiation, although low-dose CT techniques are used in the non-enhanced CT. Dual-energy CTA is an immediate automatic bone removal CTA technique that offers the advantage that images from a single CT acquisition can be used to remove bone structures. This technique enables simultaneous dual-energy image acquisition in the same phase of contrast enhancement, which reduces the radiation dose. The limitation of dual-energy CT is that it is not widely available and it requires more expensive hardware (24).

A number of factors should be taken into consideration in the interpretation of the present results. First, homogeneity tests revealed moderate heterogeneity in the sensitivity of the selected studies. The potential sources of variability among the studies were variations in the quality scores, the year that the study was published, the study design and the blinding method used. Secondly, 3 studies were excluded due to the data not enabling the reconstruction of the required 2×2 count tables. It is also possible that the search of the literature did not identify all the eligible studies, but the random omission of studies is less likely to have caused a systematic bias. Although no significant publication bias was suggested by the funnel plot and regression test, unnoticed publication bias may still have been present. Thirdly, the included studies were limited to populations with a high disease prevalence. The extrapolation of the results of the meta-analysis to screening populations with a disease prevalence that is inherently lower could introduce bias. Finally, the number of studies included in this meta-analysis was relatively small; however, in a previous systematic review (29) investigating the characteristics of meta-analyses and their included studies in the Cochrane Database, it was revealed that, irrespective of the medical field, relatively few studies are typically eligible for meta-analysis for all outcomes and interventions covered by the Cochrane reviews. Furthermore, the methodological quality of the studies included in a meta-analysis has a greater impact on the estimated effects than the numbers of included studies (30,31). QUADAS assessment revealed a high overall quality of the studies included in the present meta-analysis. In conclusion, the results of this study show that subtraction CTA is a highly sensitive, specific and non-invasive imaging method for the diagnosis and evaluation of intracranial aneurysms.

References