Evaluating Brain Image Registration Algorithms

In the well-known Klein comparative study (Klein et al., 2009 (link)), 14 image registration algorithms were evaluated based on performance on publicly available labeled brain data. For our evaluation, we used these same data. Specifically, we used the data sets denoted as:

CUMC12

IBSR18

LPBA40

MGH10

which are available for download from Arno Klein's website¹⁶.
The number of subjects per cohort is provided in the denotation. Table 1 summarizes core information about the data sets used. Further details of these first four labeled brain data (e.g., labeling protocol, data sources) are given in Klein et al. (2009 (link)). We also include the labeled brain data provided at the MICCAI 2012 Grand Challenge and Workshop on Multi-Atlas Labeling¹⁷ which we denote as MAL35. This T1-weighted MRI data set consists of 35 subject MRIs taken from the Oasis database¹⁸. The corresponding labels were provided by Neuromorphometrics, Inc¹⁹. under academic subscription.
Comparative evaluation of the two SyN registration approaches was performed within each cohort using a “pseudo-geodesic” approach. Instead of registering every subject to every other subject within a data set, we generated the transforms from each subject to a cohort-specific shape/intensity template. Not only does this reduce the computational time required for finding the pairwise transforms between subjects but prior work has demonstrated improvement in registration with this approach over direct pairwise registration (Klein et al., 2010a (link)). Since the two algorithms have been implemented within the same framework, all registration parameters are identical (i.e., linear registration stage parameters, winsorizing values, etc.) except for the parameters governing the smoothing of the gradient field.
The cohort templates were built using the ANTs script antsMultivariateTemplateConstruction.sh which is a multivariate implementation of the work described in Avants et al. (2010 (link)). Canonical views for each of the five templates used for this study are given in Figure 2. Since calculation of the transform from each subject to the template also includes generation of the corresponding inverse transform, the total transformation from a given subject to any other is determined from the composition of transforms mapping through the template. An example illustration of the geodesic approach is given in Figure 3.
Additionally, we refined the labelings for each subject of each cohort using the multi-atlas label fusion algorithm (MALF) developed by Wang et al. (2013 (link)) which is also distributed with ANTs. For a given subject within a data set, every other subject was mapped to that subject using the pseudo-geodesic transform. The set of transformed labelings were then used to determine a consensus labeling for that subject. This was to minimize the obvious observer dimensionality artifacts where manual raters observe and label in a single dimension at a time. This is most easily seen in the axial or sagittal views of the different cohorts as labelings were done primarily in the coronal view (see Figure 4). We include both sets of results. This provides two sets of labels per subject for evaluation²⁰.