Optimism

Prediction accuracy with parameters trained by WebAUGUSTUS and by human experts was measured using three different data sets. For optA, the genome of the insect Drosophila melanogaster (assembly BDGP R5/dm3) and 818 005 ESTs from the same species that were obtained from the National Center for Biotechnology Information (NCBI) were used. OptB was evaluated using the genome of the plant Arabidopsis thaliana (assembly TAIR 10) and 35 375 protein sequences of the same species that were obtained from NCBI. OptC was evaluated using the genome of the worm Caenorhabditis elegans and 18 555 training gene structures retrieved from Wormbase (16 (link)).
To avoid an overly optimistic performance estimate for the new genes, the chromosomes of all genomes [for fly and plant downloaded from the UCSC Genome Browser database (17 (link))] were split into two parts in such a way that ∼50% of the genes were located on the first half, and the remaining genes were located on the second half. The second part of all chromosomes was used as a genomic input sequence for training AUGUSTUS, whereas the first part served for accuracy assessment opf gene predictions.
For D.melanogaster, protein coding genes from FlyBase (18 (link)), for A. thaliana, protein coding genes from TAIR 10 (19 (link)) and for C. elegans, protein coding genes from Wormbase were used as a reference annotation for measuring accuracy.
The exact source of all data sets and the files used for the actual experiments are described in detail in Supplementary Materials, section Supplementary Methods: Data Sets.
Commonly used measures of accuracy (measured in percent) in gene prediction are

where TP stand for true positives, i.e. the number of predicted features that agree with the gold-standard reference, FN stands for false negatives, i.e. the number of features that were overseen by the predictor and FP stands for false positives, i.e. the number of features that were predicted but not in agreement with the reference annotation.
Sensitivity and specificity were measured for the features gene (i.e. only a gene structure that was predicted correctly including the exact positions of all CDS exons was counted as TP), exon (i.e. only exons that were predicted correctly were counted as TP) and nucleotide (i.e. every correctly predicted nucleotide was counted as TP).

The primary outcome measure was mental well-being as measured using the self-completion Warwick-Edinburgh Mental Well-being Scale (WEMWBS) [19 (link)]. This 14-item instrument has been validated for the UK population and adopted by the Scottish Health Survey and the Health Survey for England. It asks respondents to read statements about feelings and thoughts and to choose the response (a 5-point scale ranging from none of the time to all of the time) that best describes their experience over the previous two weeks. Example items are “I’ve been feeling optimistic about the future” and “I’ve been thinking clearly.” It has been shown to have good content validity and shows high correlations with other scales of mental health and well-being. It has a near-normal population distribution, with no ceiling effects.
Secondary outcomes were self-completed Center for Epidemiologic Studies Depression scale (CES-D) depression scores, Generalized Anxiety Disorder 7-item (GAD-7) anxiety scores, EuroQol Group 5-Dimension Self-Report Questionnaire (EQ-5D) quality of life scores, physical activity (self-reported frequency of exercise), and use of health services (self-reported general practitioner consultations or hospital visits). All outcomes were measured at the start of the trial (baseline before the intervention), immediately following the intervention (6 weeks after baseline), and 6 weeks after the intervention was finished (12 weeks after baseline).

This study used the R version 4.0.2 software (The R Foundation for Statistical Computing, Vienna, Austria) for data analysis, including (1) Chi-square and t-tests performed to confirm the differences according to the CEA category (CEA < 3 or CEA ≥ 3 ng/mL); (2) binary multivariable logistic regression used to identify the factors affecting the risk of CEA ≥ 3; (3) to measured effects of data discrepancies. The dataset was randomly divided into two subsets using the Caret R package, with 80% of the data (n = 5036) in the training subset and the remaining 20% (n = 1259) in the validation set. In the training cohort, significant variables (p < 0.05) were selected for binary multivariable logistic regression analysis in the univariate analysis. The model of the training cohort used backward elimination processes to predict the risk of CEA levels ≥ 3 ng/mL. The fitted model was applied to the training and validation subsets. The probability of CEA levels ≥ 3 was calculated based on the beta coefficients of the training subset. The area under the receiver operating characteristic (ROC) curve (AUC) values of the training and validation datasets were calculated using the pROC R package, and (4) To evaluate overfitting, the logistic regression model was fitted to the 1000 bootstrap samples, and the corresponding values for the AUC were calculated. The results were averaged to provide a final bootstrap estimate for AUC optimism. The differences in the values for the averaged AUC and training subset AUC provided an estimate of optimism.

To measure athlete leadership at the network level, athletes were asked to rate the leadership effectiveness for each of their teammates. A one item statement was used for each of the six dimensions of the DTLI and were scored on a scale from 1 (very poorly) to 5 (very well). These one-item statements were derived from the definitions of each of the DTLI dimensions. For inspirational motivation, participants were asked “How well does each member of your team energize you by presenting an optimistic view of the future concerning the team’s goals?.” For appropriate role model, participants were asked “How well does each member of your team serve as a role model for you?.” For fostering acceptance of group goals, participants were asked “How well does each member of you team cooperate with you in working towards the team’s goals?.” For high performance expectations, participants were asked “How well does each member of your team stress the importance of striving for excellence by having high personal performance standards?” For intellectual stimulation, participants were asked “How well does each member of your team challenge you to view problems from different perspectives.” For individualized consideration, participants were asked “How well does each member of your team show an interest in your own development as a player on this team?” An overall score for network athlete leadership for each participant was calculated by calculating the sum of these six dimensions.