Evaluating Automated Immunization Schedules

We implemented a sequential exploratory mixed methods study design with a quantitative component preceding the qualitative interviews. In the quantitative component, the children’s demographic details and immunization history were recorded by trained study staff using iDSS. The iDSS used this information to formulate an age-appropriate immunization schedule for the children, and it was recorded on the back end and was not visible to the study staff (or vaccinators). Simultaneously, the study staff also captured the manually constructed immunization schedules determined by the vaccinators as indicated on the child’s government-issued immunization cards. Through this process, we were able to capture both the iDSS and vaccinator schedules simultaneously for the same child (antigen doses). We used this information to assess the diagnostic accuracy of the iDSS algorithm by comparing the age-appropriate immunization schedules constructed by the iDSS for children aged 0 to 23 months with the gold standard of evaluation (WHO-recommended EPI schedule constructed by a vaccine expert). We also independently compared the vaccine schedules constructed manually by the vaccinators for the same children with the gold standard. This allowed us to generate preliminary evidence of MOVs resulting from inaccurate vaccination schedules constructed by vaccinators. This phase was followed by a qualitative phase in which the vaccinators were provided with iDSS-enabled study phones. After vaccinators had a chance to use the iDSS, we conducted in-depth interviews with vaccinators at the participating immunization centers regarding their experience of using the iDSS, its perceived utility, and acceptability.