Tooth Attrition

The Cochrane RoB Tool was the starting-point for developing an RoB tool for experimental animal studies. The Cochrane RoB Tool assesses the risk of bias of RCTs and addresses the following types of biases: selection bias, performance bias, attrition bias, detection bias and reporting bias [9 (link)]. The items in the Cochrane RoB Tool that were directly applicable to animal experiments were adopted (Table 2: items 1, 3, 8, 9 and 10).
To investigate which items in the tool might require adaptation, the differences between randomized clinical trials and animal intervention studies were set out (Table 1). Then we checked whether aspects of animal studies that differed from RCTs could cause bias in ways that had not yet been taken into account in the Cochrane RoB tool. Finally, the quality assessments of recent systematic reviews of experimental animal studies were examined to confirm that all aspects of internal validity had been taken into consideration in SYRCLE’s RoB tool.
To enhance transparency and applicability, we formulated signaling questions (as used in the QUADAS tool, a tool to assess the quality of diagnostic accuracy studies [15 (link),16 (link)]) to facilitate judgment. In order to obtain a preliminary idea of inter-observer agreement for each item in the RoB tool, Kappa statistics were determined on the basis of 1 systematic review including 32 papers.

Missing data are a rule rather than an exception in quantitative research. Enders (
2003 (link)
) stated that a missing rate of 15% to 20% was common in educational and psychological studies. Peng et al. (
2006
) surveyed quantitative studies published from 1998 to 2004 in 11 education and psychology journals. They found that 36% of studies had no missing data, 48% had missing data, and about 16% cannot be determined. Among studies that showed evidence of missing data, 97% used the listwise deletion (LD) or the pairwise deletion (PD) method to deal with missing data. These two methods are ad hoc and notorious for biased and/or inefficient estimates in most situations (
Rubin 1987
;
Schafer 1997
). The APA Task Force on Statistical Inference explicitly warned against their use (
Wilkinson and the Task Force on Statistical Inference 1999 (link)
p. 598). Newer and principled methods, such as the multiple-imputation (MI) method, the full information maximum likelihood (FIML) method, and the expectation-maximization (EM) method, take into consideration the conditions under which missing data occurred and provide better estimates for parameters than either LD or PD. Principled missing data methods do not replace a missing value directly; they combine available information from the observed data with statistical assumptions in order to estimate the population parameters and/or the missing data mechanism statistically.
A review of the quantitative studies published in Journal of Educational Psychology (JEP) between 2009 and 2010 revealed that, out of 68 articles that met our criteria for quantitative research, 46 (or 67.6%) articles explicitly acknowledged missing data, or were suspected to have some due to discrepancies between sample sizes and degrees of freedom. Eleven (or 16.2%) did not have missing data and the remaining 11 did not provide sufficient information to help us determine if missing data occurred. Of the 46 articles with missing data, 17 (or 37%) did not apply any method to deal with the missing data, 13 (or 28.3%) used LD or PD, 12 (or 26.1%) used FIML, four (or 8.7%) used EM, three (or 6.5%) used MI, and one (or 2.2%) used both the EM and the LD methods. Of the 29 articles that dealt with missing data, only two explained their rationale for using FIML and LD, respectively. One article misinterpreted FIML as an imputation method. Another was suspected to have used either LD or an imputation method to deal with attrition in a PISA data set (
OECD 2009
;
Williams and Williams 2010 (link)
).
Compared with missing data treatments by articles published in JEP between 1998 and 2004 (
Table 3.1 in Peng et al. 2006
), there has been improvement in the decreased use of LD (from 80.7% down to 21.7%) and PD (from 17.3% down to 6.5%), and an increased use of FIML (from 0% up to 26.1%), EM (from 1.0% up to 8.7%), or MI (from 0% up to 6.5%). Yet several research practices still prevailed from a decade ago, namely, not explicitly acknowledging the presence of missing data, not describing the particular approach used in dealing with missing data, and not testing assumptions associated with missing data methods. These findings suggest that researchers in educational psychology have not fully embraced principled missing data methods in research.
Although treating missing data is usually not the focus of a substantive study, failing to do so properly causes serious problems. First, missing data can introduce potential bias in parameter estimation and weaken the generalizability of the results (
Rubin 1987
;
Schafer 1997
). Second, ignoring cases with missing data leads to the loss of information which in turn decreases statistical power and increases standard errors(
Peng et al. 2006
). Finally, most statistical procedures are designed for complete data (
Schafer and Graham 2002 (link)
). Before a data set with missing values can be analyzed by these statistical procedures, it needs to be edited in some way into a “complete” data set. Failing to edit the data properly can make the data unsuitable for a statistical procedure and the statistical analyses vulnerable to violations of assumptions.
Because of the prevalence of the missing data problem and the threats it poses to statistical inferences, this paper is interested in promoting three principled methods, namely, MI, FIML, and EM, by illustrating these methods with an empirical data set and discussing issues surrounding their applications. Each method is demonstrated using SAS 9.3. Results are contrasted with those obtained from the complete data set and the LD method. The relative merits of each method are noted, along with common features they share. The paper concludes with an emphasis on assumptions associated with these principled methods and recommendations for researchers. The remainder of this paper is divided into the following sections: (1) Terminology, (2) Multiple Imputation (MI), (3) Full Information Maximum-Likelihood (FIML), (4) Expectation-Maximization (EM) Algorithm, (5) Demonstration, (6) Results, and (6) Discussion.

Recruitment for the semistructured interviews was conducted using purposive and snowball sampling at a large Veterans Affairs hospital in the Southeast United States. Veterans and their care partners were identified from the following sources: (1) on-site in services conducted with primary care teams, (2) web-based in services, (3) direct staff referrals, and (4) internal tracking of the report of current registrants. Potential participants were approached via phone. A subsample of the interview participants were asked to participate in the subsequent user testing phase. A subsample of three agreed to participate in user testing. One health care staff became aware of the project and volunteered to be part of the user testing for a staff perspective.
Inclusion criteria included veterans who were aged ≥18 years, who were registered My HealtheVet users, who had no cognitive impairment that prevented the use of a PC or the ability to engage in project activities, and who reported having a caregiver who assisted them with health care management. Inclusion criteria for care partners included those aged ≥18 years, who had no cognitive impairment that prevented the use of a PC or the ability to engage in project activities, and who reported providing caregiving assistance.
On the basis of qualitative sampling methods, saturation was anticipated to occur between 12 and 15 interviews for each VDT user type (ie, veteran and care partner) [22 ]. An overrecruitment strategy was used to allow for attrition. Up to 25 individuals representing each user type were recruited to ensure saturation across domains.

A PhD-level researcher systematically searched Google Scholar, PsycINFO, and PubMed databases for articles with the word “binge” in their title and the terms “ecological momentary” or “experience sampling” to find risk state descriptors with relevance to binge eating in the literature. The search resulted in 509 articles that were deduplicated and scanned for relevance. Only empirical articles reporting the results of EMA studies on binge eating were retained. A total of 262 articles were subsequently analyzed (see Multimedia Appendix 1, Figure S1 for an attrition diagram).