A novel eight-step validation method for the development of a nutrition knowledge questionnaire was designed based on an extensive review of the literature and used to validate this questionnaire [35 ]. The steps include: (1) Definition of Sports Nutrition Knowledge (2) Generation of items to represent sports nutrition knowledge (3) Choice of scoring system (4) Assessment of content validity by panel of experts (5) Assessment of face validity by student athletes (6) CTT analysis: Removal of items on the basis of item difficulty, item discrimination and distractor utility (7) Rasch analysis: Assessment of dimensionality and removal of item on the basis of not meeting assumptions that difficult questions are less likely to be answered correctly, and well-scoring participants are more likely to answer individual items correctly (8) Assessment of construct validity by comparing nutrition and non-nutrition students; assessment of test-retest reliability (consistency over time) by assessing correlation of test on two attempts; and re-checking of steps six and seven. The steps that make this methodology novel are the quantitative assessment of content validity, the assessment of distractor utility (how feasible incorrect multiple choice options are) and the inclusion of Rasch analysis.
Figure 1 provides a summary of the methods and results.

Flow chart of 8-step methadology used to develop and validate the Nutrition for Sport Questionnaire (UNSQ). * Content Validity = the measure covers all relevant topics related to sports nutrition. † CVI = Number of experts who rated an item ‘very relevant’ or ‘relevant’ divided by total number of experts; > 0.78 is adequate. ‡ Face Validity = the measure, on face value is an adequate reflection of sports nutrition. § Difficulty index = frequency with which items were answered correctly; <20% = too hard; >80% = too easy. ǁ Discrimination index = average score of top 10% of participants minus average score of bottom 10% of participants; > 0.3 is adequate. ¶ Distractor utility = frequency with which each multi-choice option is selected; > 5% = effective distractor. **Fit residuals between −2.5 and 2.5 indicate observed = expected responses. ††DIF assessed using ANOVA; non-significant p-value = no differences in response pattern based on participant characteristics; ‡‡ Disordered thresholds are assessed graphically. §§ Perc5% statistic <5% = scale is unidimensional (assessing one concept). ǁ ǁ SD of 0 and Mean of 1for the overall item/person interaction = perfect fit to Rasch model; a SD > 1.5 = misfit. ¶¶ Significant differences in known-group comparison scores = construct validity (questionnaire test what it is supposed to). *** Pearson’s r > 0.7 = test-retest reliability (stability overtime). ††† KR-20 > 0.7 = Internal reliability (consistency in items)

Free full text: Click here