Around a third to half of all systematic reviews do not use meta-analysis at all,23 (link),24 (link) and more do not use meta-analysis for all outcomes. Reasons why meta-analysis may not be possible are varied, but one reason may be when the included primary studies do not provide the necessary data required for a meta-analysis. For example, the studies may report statements about the direction of effect without an effect estimate, or report effect estimates without measures of variance.
For the first time, Chapter 12 of the Handbook presents a number of alternative statistical synthesis methods for this scenario, including the use of summary statistics to describe the observed range of intervention effect estimates, vote counting based on the direction of effect, or combining P values. Companion visual displays for these synthesis methods are also described, including box plots, harvest plots and albatross plots.
In some cases, synthesis may be inappropriate, for example where the included studies are at a high risk of bias or there are concerns about missing evidence. In such cases, structured tabulation or plotting the results without synthesis may be used for presentation.25
Although these alternative synthesis methods yield results that are more limited for decision making in comparison to meta-analysis, they provide additional options for review authors when the required data for meta-analysis are not available. These methods may enable authors to make the most of the available data and provide a structured approach where authors may have felt limited to text-based descriptions of the findings of individual studies, which can become rapidly unwieldy where large numbers of studies or complex interventions are involved. The recently updated PRISMA26 (link) and Synthesis Without Meta-analysis (SWiM)27 reporting guidelines may be of additional assistance to authors unsure of how to specify the use of these methods at the protocol stage of the review, and how to completely report which methods have been used in practice.
For the first time, Chapter 12 of the Handbook presents a number of alternative statistical synthesis methods for this scenario, including the use of summary statistics to describe the observed range of intervention effect estimates, vote counting based on the direction of effect, or combining P values. Companion visual displays for these synthesis methods are also described, including box plots, harvest plots and albatross plots.
In some cases, synthesis may be inappropriate, for example where the included studies are at a high risk of bias or there are concerns about missing evidence. In such cases, structured tabulation or plotting the results without synthesis may be used for presentation.25
Although these alternative synthesis methods yield results that are more limited for decision making in comparison to meta-analysis, they provide additional options for review authors when the required data for meta-analysis are not available. These methods may enable authors to make the most of the available data and provide a structured approach where authors may have felt limited to text-based descriptions of the findings of individual studies, which can become rapidly unwieldy where large numbers of studies or complex interventions are involved. The recently updated PRISMA26 (link) and Synthesis Without Meta-analysis (SWiM)27 reporting guidelines may be of additional assistance to authors unsure of how to specify the use of these methods at the protocol stage of the review, and how to completely report which methods have been used in practice.
