Ensemble Modeling for Cause-Specific Mortality

The GBD Cause of Death Ensemble model (CODEm) systematically tested and combined results from different statistical models according to their out-of-sample predictive validity. Results are incorporated into a weighted ensemble model as detailed in appendix 1 (section 3.1) and below. For GBD 2017, CODEm was used to estimate 192 causes of death (appendix 1 section 7). To predict the level for each cause of death, we used CODEm to systematically test a large number of functional forms and permutations of covariates.¹⁸ (link) Each resulting model that met the predetermined requirements for regression coefficient significance and direction was fit on 70% of the data, holding out 30% for cross-validation (appendix 1 section 3.1). Out-of-sample predictive validity of these models was assessed by use of repeated cross-validation tests on the first 15% of the held-out data. Various ensemble models with different weighting parameters were created from the combination of these models, with the highest weights assigned to models with the best out-of-sample prediction error for trends and levels, as detailed in appendix 1 (section 7). Model performance of these ensembles was assessed against the root-mean squared error (RMSE) of the ensemble model predictions of the log of the age-specific death rates for a cause, assessed with the same 15% of the data. The ensemble model performing best was subsequently selected and assessed against the other 15% of the data withheld from the statistical model building. CODEm was run independently by sex for each cause of death. A separate model was run for countries with 4-star or greater VR systems to avert uncertainty inflation from more heterogeneous data. The distribution of RMSE relative to cause-specific mortality rates (CSMRs) at Level 2 of the GBD hierarchy shows that model performance was weakest for causes of death with comparatively low mortality rates (figure 2; appendix 2), while models for more common causes of death such as stroke, chronic obstructive pulmonary disease, and self-harm and interpersonal violence generally had low RMSE.Figure 2

Out-of-sample model performance for CODEm models and age-standardised cause-specific mortality rate by Level 1 causes

Model performance was defined by the root-mean squared error of the ensemble model predictions of the log of the age-specific death rates for a cause with 15% of the data held out from the statistical model building. The figure shows the association between the root-mean squared error and the log of the CSMR, aggregated over 1980–2017. Each point represents one CODEm model specific for model-specific age ranges and sex. Circles denote models run with all locations. Triangles denote models run on only data-rich locations. Colours denote the Level 1 cause categories. Open circles and triangles denote models that were run with restricted age groups of less than 30 years. CODEm=Cause of Death Ensemble model. CSMR=cause-specific mortality rate.