Assessing Risk of Bias in Prognostic Studies

To assess the RoB of the included papers, the QUIPS electronic spreadsheet (in MS Excel) as provided by Hayden et al. [11 (link)] was used. The two raters independently inserted relevant information from each paper in their own electronic assessment spreadsheet. The raters were able to make specific notes on quality issues for each prompting item. Each rater used only one MS Excel sheet for each paper, although several prognostic factors and outcomes could have been evaluated in the same paper. Hence, for each domain, the raters judged the six domains on the overall quality of the paper. The authors’ notes were used later during our peer-group discussions for reaching agreement (on the overall quality of the study) when summarizing the level of evidence for every separate prognostic factor.
For QUIPS, there are no rules available that indicate how the researcher should classify the overall RoB of a paper or, in other words, how to summarize the RoB of all six domains into one overall rating on paper level, but it is recommended against computing summated scores for overall study quality [14 (link)]. In systematic reviews and/or meta-synthesis, however, it is recommended to include a table of included papers in which each paper is classified as having high, moderate or low RoB. It became thus evident that some sort of categorization of the papers was necessary to describe the included papers for our synthesis after finalizing all RoB assessments [3 ]. We based this categorization on the following criteria: If all domains were classified as having low RoB, or up to one moderate RoB, then this paper was classified as low RoB (green). If one or more domains were classified as having high RoB, or ≥ 3 moderate RoB, then this paper was classified as high RoB (red). All papers in between were classified as having moderate RoB (yellow). This categorization was a result of a continuous discussion between the authors.