In order to assess the face validity of our modification to the ALE approach we also performed ALE meta-analyses on two simulated datasets. It should be noted that the “studies” included into these datasets do not correspond to any real data as published in the literature. Rather each study solely refers to a set of individual foci, i.e., MNI coordinates, which were generated in order to simulate situations occurring in meta-analyses.
The first simulated dataset consists of 25 studies. Each of these studies is supposed to having investigated 12 subjects in order to avoid confounding effects of different sample sizes. For every study, we set one focus on the inferior frontal gyrus corresponding to BA 44. Hence, this region is the location of the “true” activation, which is to be revealed by the meta-analyses. Furthermore, a single one out of the 25 studies also features an activation in the inferior parietal lobe (IPL). For this activation, however, 10 individual foci are given, corresponding to a situation where individual local maxima are listed in a very detailed fashion. This analysis aims at revealing the distinction between fixed- and random-effects analyses. Fixed-effects analyses as implemented in classical ALE assess the convergence between individual foci. It should therefore reveal a significant effect in the IPL because 10 foci cluster closely within this area. In contradistinction, this location should not become significant in a random-effects analysis, as all of these foci were reported in the same study and the object of inference is to reveal a convergence across studies. Both methods, however, should identify the clustering of activations in the inferior frontal gyrus.
The second dataset also consists of 25 studies, and again features a true convergence of the reported activations in BA 44. Out of these 25 studies, four are assumed to have investigated 30 subjects each. Due to the higher reliability resulting from such larger samples, these four foci all cluster very tightly around a presumed true location of the effect. The remaining 21 studies, however, only examined four subjects, i.e., had very small sample sizes. Consequently, the locations of the reported foci are simulated to be more variable (due to the larger influence of sampling effects). This analysis aims at testing the explicit variance model employed in the revised ALE approach. As outlined above, the between-subject variance enters the variance model scaled by the sample size resulting in smaller FWHMs for studies investigating larger samples. Consequently the latter studies should have increased localising power in the ALE meta-analysis. In the present simulation, we would hence expect that the results obtained from the revised ALE algorithm would be less influenced by the foci obtained from the smaller studies and, therefore, more confined to the location of the foci reported in the four larger studies.
To each of the individual studies in both simulated meta-analyses, 10 further foci are added, which were randomly (and independently across studies) allocated to grey matter voxels. In real datasets, these foci would correspond to activations evoked by other components of the respective tasks. In the context of these meta-analyses, however, they represent noise, as there is no convergence between them. Both datasets are then analysed using the original ALE algorithm and its revised version in the same manner as the experimental data described above.
The first simulated dataset consists of 25 studies. Each of these studies is supposed to having investigated 12 subjects in order to avoid confounding effects of different sample sizes. For every study, we set one focus on the inferior frontal gyrus corresponding to BA 44. Hence, this region is the location of the “true” activation, which is to be revealed by the meta-analyses. Furthermore, a single one out of the 25 studies also features an activation in the inferior parietal lobe (IPL). For this activation, however, 10 individual foci are given, corresponding to a situation where individual local maxima are listed in a very detailed fashion. This analysis aims at revealing the distinction between fixed- and random-effects analyses. Fixed-effects analyses as implemented in classical ALE assess the convergence between individual foci. It should therefore reveal a significant effect in the IPL because 10 foci cluster closely within this area. In contradistinction, this location should not become significant in a random-effects analysis, as all of these foci were reported in the same study and the object of inference is to reveal a convergence across studies. Both methods, however, should identify the clustering of activations in the inferior frontal gyrus.
The second dataset also consists of 25 studies, and again features a true convergence of the reported activations in BA 44. Out of these 25 studies, four are assumed to have investigated 30 subjects each. Due to the higher reliability resulting from such larger samples, these four foci all cluster very tightly around a presumed true location of the effect. The remaining 21 studies, however, only examined four subjects, i.e., had very small sample sizes. Consequently, the locations of the reported foci are simulated to be more variable (due to the larger influence of sampling effects). This analysis aims at testing the explicit variance model employed in the revised ALE approach. As outlined above, the between-subject variance enters the variance model scaled by the sample size resulting in smaller FWHMs for studies investigating larger samples. Consequently the latter studies should have increased localising power in the ALE meta-analysis. In the present simulation, we would hence expect that the results obtained from the revised ALE algorithm would be less influenced by the foci obtained from the smaller studies and, therefore, more confined to the location of the foci reported in the four larger studies.
To each of the individual studies in both simulated meta-analyses, 10 further foci are added, which were randomly (and independently across studies) allocated to grey matter voxels. In real datasets, these foci would correspond to activations evoked by other components of the respective tasks. In the context of these meta-analyses, however, they represent noise, as there is no convergence between them. Both datasets are then analysed using the original ALE algorithm and its revised version in the same manner as the experimental data described above.