A retrospective analysis of 18,642 participants with brain tumors who underwent craniotomy between 2012 and 2015 was performed; the information regarding these patients was retrieved from the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) database (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7498000/, S1 Data). The ACS NSQIP is a validated, prospectively collected, publicly available, peer-controlled database of a random sample of outpatients and inpatients undergoing nontrauma surgery at ~400 community and academic hospitals across the US. The identities of the patients were encrypted as nontraceable codes to ensure participant privacy. Variables at baseline were included as screening variables in the prediction model in the present study. The dependent variable was postoperative 30-day mortality (dichotomous variable: 0=nonpostoperative 30-day mortality; 1=postoperative 30-day mortality).

Zhang et al (24) previously published an article titled ‘Sepsis and septic shock after craniotomy: Predicting a significant patient safety and quality outcome measure’ and uploaded the original data to the ACS NSQIP database. The uploaded data are available for use in secondary analyses without infringement on the authors' rights and the copyright statement.

The following variables were extracted for the present study according to the previous literature and our clinical experience: i) Continuous variables, including body height, body weight and indicators of preoperative blood test results [hematocrit (HCT), blood urea nitrogen (BUN), white blood cell (WBC) count, creatine (Cr) and platelet (PLT) count], and ii) categorical variables, including sex, ethnicity, age range, diabetes status, smoking status, year of operation, dyspnea, functional health status, ventilator dependence, severe chronic obstructive pulmonary disease (COPD), congestive heart failure (CHF), hypertension, renal failure, preoperation transfusions, dialysis, disseminated cancer, preoperative systemic sepsis, open wound infection, steroid use for chronic conditions, >10% loss of body weight in the last 6 months, bleeding disorders, emergency cases, wound classification and American Society of Anesthesiologists (ASA) physical status classification. More elaborate details were presented in the original study (24). The body mass index (BMI) was calculated as weight in kilograms divided by height in meters squared (kg/m²).

The number of participants with missing BMI (weight and height), functional health status, Na, BUN, Cr, WBC, HCT, PLT and ASA data was 730 (3.92%), 90 (0.48%), 798 (4.28%), 1,532 (8.22%), 709 (3.8%), 592 (3.18%), 440 (2.36%), 579 (3.11%) and 166 (0.89%), respectively. Multiple imputation techniques are widely accepted as appropriate methods for handling missing data (25). This method was used to input missing values for the extracted variables in the present study. The imputation model included BMI; functional health status; Na, BUN, and Cr levels; WBC count; HCT level; PLT count; and ASA classification. Missing data analysis procedures used missing-at-random assumptions (26).

The primary outcome variable was postoperative 30-day mortality. The NSQIP was used to track mortality for the first 30 postoperative days (24).

A training dataset (patients who underwent craniotomy in 2012 and 2013) and an external validation dataset (those who underwent craniotomy in 2014 and 2015) were generated from the initial study population. The training dataset was used to establish the model and the external validation dataset was used for independent evaluation of the preliminary model's performance.

Baseline characteristics are expressed as the mean ± standard deviation (normal distribution) or the median (interquartile range) (skewed distribution) for continuous variables and as the frequency and percentage for categorical variables. Two-samples t-tests were applied to analyze differences between the training and validation cohorts for normally distributed continuous variables. Wilcoxon rank-sum tests were used for nonnormally distributed continuous variables, and chi-square test or Fisher's exact test was used for categorical variables. The baseline characteristics of the training and validation cohorts stratified were also presented with stratification by incident 30-day mortality. Univariate and multivariate analyses were also performed to identify potential risk factors of 30-day postoperative mortality after craniotomy for brain tumors.

To construct a reliable and simple risk prediction model, two rounds of variable screening were conducted. The least absolute shrinkage and selection operator (LASSO) method is frequently used for domains with very large datasets and is suitable for the reduction of high-dimensional data (27). This dataset was used to select the most useful predictive candidates from the training dataset. Candidates with nonzero coefficients in the LASSO regression model were selected (28). A second screening round was performed based on the LASSO model's identified variables. First, all of the risk factors were applied to construct a full logistic regression model. Second, a backward step-down selection process was conducted according to the Akaike information criterion to establish a parsimonious model (a stepwise logistic proportional hazards model) (29). Third, according to the multivariable fractional polynomial (MFP) algorithm, an iterative approach was used to determine the significant variables and functional form via backward elimination to establish a stable model (MFP model) in the real world (30). Considering that there were fewer variables in the stepwise model and that the predictive performance was relatively good, the stepwise model was selected for further analysis.

To evaluate and compare the discriminatory power of these prediction models, the receiver operating characteristic (ROC) curve was plotted and the area under the ROC curve (AUC) with 95% confidence intervals (CIs) was calculated for the training dataset and validation dataset. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), negative likelihood ratio (NLR) and diagnostic odds ratio (DOR) of the stepwise model, which were calculated according to standard definitions, were simultaneously presented. A prediction formula was obtained from the stepwise logistic proportional hazards model. The nomogram was based on proportionally converting each regression coefficient in the multivariate logistic regression to a 0- to 100-point scale (31). The effect of the variable with the highest β coefficient (absolute value) was assigned 100 points. The points were added across independent variables to derive total points, which were converted to predicted probabilities of postoperative 30-day mortality. The nomogram score was a numeric value representing the prediction model score of the individual patient. The sensitivity and specificity for predicting 30-day mortality were different at different cutoff values of the nomogram scores. In addition, a calibration plot for the probability of 30-day mortality was generated to assess the accuracy of the nomogram (32).

The associated risk factors for 30-day mortality in the stepwise model were also categorized according to clinical cutoff points to create the score model of 30-day mortality. These risk factors, which were treated as categorical variables, were included in the stepwise logistic proportional hazards model and a new β coefficient was derived. The scoring system was developed based on regression coefficients multiplied by 2 and rounded to the nearest integer to derive the weights of the scores (33). This scoring system was subsequently presented as a questionnaire form that can be easily used by health personnel in primary care. The total score was divided into four risk categories: Low, intermediate, high and extremely high risk categories. The performance of our risk score model for predicting postoperative 30-day mortality was also tested by analyzing the performance of each risk factor in the model and its optimal cutoff for predicting postoperative 30-day mortality based on ROC curves. All of the results are reported according to the TRIPOD statement (34).

All of the analyses were performed with the statistical software packages R (http://www.R-project.org; The R Foundation) and EmpowerStats (http://www.empowerstats.com; X&Y Solutions, Inc.). All of the tests were 2-sided, with P<0.05 considered to indicate statistical significance.

The current study included 18,642 adult participants (47.40% of whom were men) (Table SI). The age distributions were 16.40% (18–40 years), 41.53% (41–60 years), 38.80% (61–80 years) and 3.27% (>81 years). The mean BMI was 28.69±6.72 kg/m², the mean Na concentration was 138.62±3.22 mmol/l, the mean BUN concentration was 17.39±8.31 mg/dl, the mean Cr concentration was 0.87±0.45 mg/dl, the mean WBC count was 9.50±4.48×10⁹/l, the mean HCT level was 40.35±4.81%, and the mean PLT count was 243.4±76.90×10⁹/l. The postoperative 30-day mortality of the included participants was 2.46% (458/18,642).

Table I shows the basic demographic, anthropological and clinical information for the eligible participants. The participants were assigned to two groups based on the year of surgery: The training dataset (2012–2013) and the validation dataset (2014–2015). For numerous baseline characteristics, although the differences between the training cohort and the validation cohort were statistically significant due to the large sample size (P<0.05), they were not clinically significant.

Table II shows the baseline characteristics of patients with nonpostoperative 30-day mortality and postoperative 30-day mortality in the training and validation datasets. The participants with postoperative 30-day mortality had higher BUN levels and WBC counts in the training and validation cohorts (all P<0.01). By contrast, the participants who died within 30 days postoperatively had lower Na concentrations, HCT levels and PLT counts (all P<0.05).

The results of the univariate and multivariate analyses using a binary logistic regression model are presented in Table SII. The univariate analysis showed that female sex (OR=0.649), age range (41–60 years) (OR=2.682), age range (61–80 years) (OR=4.940), age (>81 years) (OR=14.902), BMI (OR=0.985), diabetes (noninsulin-dependent) (OR=1.552), diabetes (insulin-dependent) (OR=2.618), dyspnea (moderate exertion) (OR=2.333), dyspnea (moderate exertion) (OR=2.333), functional health status (partially dependent) (OR=4.032), functional health status (totally dependent) (OR=7.211), ventilator dependence (OR=4.527), severe COPD (OR=2.525), CHF (OR=7.270), hypertension (OR=2.292), renal failure (OR=10.893), dialysis (OR=6.580), disseminated cancer (OR=2.913), open wound infection (OR=5.041), steroid use for chronic conditions (OR=2.330), >10% body weight loss in last 6 months (OR=4.255), bleeding disorders (OR=2.307), preoperative transfusions (OR=5.860), SIRS (OR=2.186), sepsis (OR=13.470), septic shock (OR=9.354), levels of Na (OR=0.910), BUN (OR=1.043), Cr (OR=1.240), WBC count (OR=1.074), HCT level (OR=0.923), PLT count (OR=0.998), emergency cases (OR=2.875) and wound classification (dirty/infected) (OR=5.506) were associated with postoperative 30-day mortality (all P<0.05).

The multivariate analysis demonstrated that female sex (OR=0.713), age range (41–60 years) (OR=1.927), age range (61–80 years) (OR=2.573), age (>81 years) (OR=6.680), functional health status (partially dependent) (OR=2.152), functional health status (totally dependent) (OR=2.982), ventilator-dependence (OR=2.402), hypertension (OR=1.374), disseminated cancer (OR=1.791), open wound infection (OR=2.297), steroid use for chronic conditions (OR=1.619), >10% body weight loss in the last 6 months (OR=2.067), sepsis (OR=3.563), Na level (OR=0.971), BUN level (OR=1.019), WBC count (OR=1.049), HCT level (OR=0.959), PLT count (OR=0.998), emergency cases (OR=2.267) and wound classification (dirty/infected) (OR=3.228) were associated with postoperative 30-day mortality (all P<0.05).

Of the clinical features, 30 indicators [BMI and preoperative blood test results (HCT, BUN, WBC, Cr, PLT), sex, ethnicity, age ranges, diabetes status, smoking status, dyspnea, functional health status, ventilator dependence, severe COPD, CHF, hypertension, renal failure, dialysis, disseminated cancer, open wound infections, steroid use for chronic conditions, >10% loss of body weight in the last 6 months, bleeding disorders, preoperative transfusions, preoperative systemic sepsis, emergency cases, wound classification and ASA physical status classification)] were reduced to 6 potential predictors based on 7,800 participants in the training dataset (Fig. 1A and B) with nonzero coefficients in the LASSO regression model. These potential predictors included the preoperative WBC count, BUN level, HCT level, age range, functional health status and disseminated cancer.

A total of three prediction models were further established based on the predictors chosen by the LASSO regression model, namely the MFP model, the full logistic proportional hazards model and the stepwise logistic regression model. In the training cohort, the AUC values of the MFP model, full model and stepwise model were 0.7983, 0.7949 and 0.7949, respectively. In the validation cohort, the corresponding AUC values of these models were 0.7423, 0.7382 and 0.7382, respectively (Fig. S1A and B). The AUCs of the three models were relatively close. Given that the stepwise model incorporated fewer risk factors, it was simpler than the MFP and full models. In addition, the stepwise model could predict the risk of postoperative 30-day mortality relatively well. Therefore, the stepwise model was selected as the optimal risk prediction model for postoperative 30-day mortality. As indicated in Table III, 6 variables were selected according to the stepwise model: WBC count (OR=1.0710, 95% CI=1.0420–1.1009), age range (41–60 years) (OR=1.7108, 95% CI=0.8032–3.6439), age range (61–80 years) (OR=2.8297, 95% CI=1.3510–5.9270), age (>81 years) (OR=8.2427, 95% CI=3.5937–18.9056), BUN (OR=1.020, 95% CI=1.008–1.031), HCT (OR=0.945, 95% CI=0.919–0.972), functional health status (partially dependent) (OR=3.0521, 95% CI=1.9820–4.7000), functional health status (totally dependent) (OR=2.9286, 95% CI=0.9864–8.6944) and disseminated cancer status (OR=2.8180, 95% CI=2.0631–3.8490). The results showed that 5 variables (excluding the level of HCT) were positively associated with postoperative 30-day mortality.

The ability of each risk factor to predict postoperative 30-day mortality was evaluated in the training and validation cohorts (Tables SIII and SIV; Fig. S2A and B). Tables SIII and SIV indicate that each risk predictor showed high accuracy in our nomogram.

A corresponding nomogram was further constructed to provide a quantitative and simple tool for predicting the risk of postoperative 30-day mortality by using the preoperative WBC count, HCT level, BUN level, age ranges, functional health status and disseminated cancer incidence (Fig. 2). Each variable in the nomogram was assigned a specific point value and the points for each variable were summed to obtain the total points, which were used to determine the probability of postoperative 30-day mortality. The algorithm for determining the risk of postoperative 30-day mortality in the stepwise model was as follows: Log (Y)=−3.90696+0.06862 × WBC (×10⁹/l) + 0.53696 × [age range (41–60)] (years) + 1.04019 × [age range (61–80)] (years) +2.10932 × [age range (>81)] (years) +1.11582 × (functional health status, partially dependent) +1.07452 × (functional health status, totally dependent) +1.03602 × (disseminated cancer) + 0.01967 × BUN (mg/dl)-0.05668 × HCT (%). Probability of 30-day mortality=1/{1+e^[-log(Y)]}.

It was also evaluated how close the predicted postoperative 30-day mortality was to the observed postoperative 30-day mortality risk for the nomogram in the training and validation cohorts. The calibration for the probability of postoperative 30-day mortality showed excellent agreement between the predicted possibility and the actual observation in both the training and validation sets (Fig. 4). These results demonstrated that the nomogram was able to accurately predict postoperative 30-day mortality in an American population.

Selected continuous variables (BUN, WBC and HCT) were converted into categorical variables according to the best threshold (Table V). Score points were assigned to each risk factor by using the model parameter estimates, after which the values were multiplied by 2 and rounded to the nearest integer. The logistic estimates for the risk variables, corresponding score points and the contributed AUC for each variable are provided in Table VI.

The resulting 30-day mortality scores ranged between a minimum of −1.5 and a maximum of 9 points and were divided into four groups according to the quartile of the total risk score as follows: Low risk (−1.5 to −1), moderate risk (−0.5 to 0.5), high risk (1 to 2) and extremely high risk (2.5 to 9) (Table VII). The observed incidence of mortality among low-risk subjects (−1.5 to −1 point) was 0.28% (2 out of 718 patients), the incidence among moderate-risk subjects was 0.73% (22 out of 3,003 patients) (−0.5 to 0.5 points), the incidence among high-risk participants was 1.17% (22 out of 1,879 patients) (1 to 2 points) and the incidence among extremely high-risk subjects was 6.90% (139 out of 2,015 patients) (Table VII).

External validation of the risk score was conducted on a cohort of 10,842 participants (those individuals who underwent craniotomies in 2014–2015). In the validation cohort, the resulting 30-day mortality scores were also divided into four groups according to the quartile of the total risk score as follows: Low risk (−1.5 to −1), moderate risk (−0.5–0.5), high risk (1–2) and extremely high risk (2.5–9). The observed incidence of postoperative 30-day mortality among low-risk participants (−1.5 to −1 point) was 0.29% (3 out of 1,051 patients), among moderate-risk participants it was 1% (41 out of 4,112 patients) (−0.5 to 0.5 points), among high-risk participants it was 2.72% (71 out of 2,609 patients) (1 to 2 points) and among extremely high-risk participants it was 5.64% (158 out of 2,797 patients) (Table VII). The incidences of death in the validation group and the modeling group were similar for each score group (Table VII), thus indicating that the scoring model had good predictive performance.

It was also calculated that the AUC values of the scoring scale model were 0.7844 (95% CI=0.7526–0.8162) and 0.7289 (95% CI=0.7012–0.7566) in the training cohort and the validation cohort, respectively (Table SV). At the best thresholds (2.25 and 1.75), the specificities were 73.54 and 65.49%, and the sensitivities were 75.14 and 68.50% for the training and validation cohorts, respectively (Table SV). The training and validation cohorts both had relatively high NPVs.