All speech data were transcribed in ELAN [39 ]. The duration between utterances was extracted automatically in milliseconds (ms). Positive durations correspond to gaps and negative durations correspond to overlaps. For the turn-taking analysis presented here we only considered cases where there was a speaker change. We do not consider whether the speaker change occurs at a TRP or not—in some cases it will, in other cases it will not (for example, where there is an apparent speaker switch for a mid-turn backchannel from another speaker which will split a single turn from a speaker into two utterances, with an intervening backchannel). Of course, this has consequences for the analysis and interpretation, however, the methodology applies equally to both PSz and C groups so differences between them are still relevant and meaningful. See also [40 ] for discussion of related issues.
Self-repairs were annotated using strongly incremental repair detection (STIR; [41 ]) which automatically detects speech repairs on transcripts. STIR is trained on the Switchboard corpus [42 ], and has been shown to be applicable to therapeutic dialogue, with high rates of correlation to human coders in terms of self-repair rate [43 ].
Hand movement was automatically extracted from the raw motion capture data. In order to control for individual variation, for each participant we extracted the movement from each of the three hand/wrist markers, and calculated the mean and standard deviation (s.d.) of movement in any direction by frame in millimetres (mm). For frames with missing markers, if this was fewer than 50 frames (frame rate 60 s−1), we imputed the missing data using a linear trajectory, otherwise left the data as missing. Following the methodology in [33 (link),44 ], to account for individual variation, for each pair of frames we calculated whether the movement between them was greater than the individual’s , for any of the three markers, and if so marked this as movement. The use of all three wrist and hand markers helps to mitigate the points where single markers dropped out, e.g. owing to occlusion. The hand movement data were imported to ELAN. Visual inspection of the data suggested that using an individual’s is generally a good proxy for hand movement. However, this is not the case where this value was very low (owing to minimal or no movement, or extreme cases of marker drop out) in which case the algorithm was oversensitive to minor non-gestural movements caused by posture shifts, for example. It was also not accurate in cases where the value of the was very high (individuals who gesture a lot), in which case the algorithm was undersensitive to genuine gestures. For this reason, we introduced a lower and upper threshold for the movement values. These were set at 2 mm per frame for the lower bound and 5 mm per frame for the upper bound. These refinements to the movement calculation result in a more reliable and sensitive index of hand movement than has been adopted in previous analyses of this corpus (e.g. [33 (link)]).
It should be borne in mind that although we believe that our automatically derived hand movement measures are a good proxy for gesture, they do not distinguish between gestural hand movement and other hand movement (e.g. scratching, fidgeting). It is also the case that the automatic hand movement annotation captures only the movement phases of a gesture—including preparation and retraction [16 ], and will not pick up any hold phases of gestures, which are known to be interactionally relevant (see e.g. [45 (link)]), particularly in respect to turn-taking.
Analyses were performed in SPSS 28.
Self-repairs were annotated using strongly incremental repair detection (STIR; [41 ]) which automatically detects speech repairs on transcripts. STIR is trained on the Switchboard corpus [42 ], and has been shown to be applicable to therapeutic dialogue, with high rates of correlation to human coders in terms of self-repair rate [43 ].
Hand movement was automatically extracted from the raw motion capture data. In order to control for individual variation, for each participant we extracted the movement from each of the three hand/wrist markers, and calculated the mean and standard deviation (s.d.) of movement in any direction by frame in millimetres (mm). For frames with missing markers, if this was fewer than 50 frames (frame rate 60 s−1), we imputed the missing data using a linear trajectory, otherwise left the data as missing. Following the methodology in [33 (link),44 ], to account for individual variation, for each pair of frames we calculated whether the movement between them was greater than the individual’s , for any of the three markers, and if so marked this as movement. The use of all three wrist and hand markers helps to mitigate the points where single markers dropped out, e.g. owing to occlusion. The hand movement data were imported to ELAN. Visual inspection of the data suggested that using an individual’s is generally a good proxy for hand movement. However, this is not the case where this value was very low (owing to minimal or no movement, or extreme cases of marker drop out) in which case the algorithm was oversensitive to minor non-gestural movements caused by posture shifts, for example. It was also not accurate in cases where the value of the was very high (individuals who gesture a lot), in which case the algorithm was undersensitive to genuine gestures. For this reason, we introduced a lower and upper threshold for the movement values. These were set at 2 mm per frame for the lower bound and 5 mm per frame for the upper bound. These refinements to the movement calculation result in a more reliable and sensitive index of hand movement than has been adopted in previous analyses of this corpus (e.g. [33 (link)]).
It should be borne in mind that although we believe that our automatically derived hand movement measures are a good proxy for gesture, they do not distinguish between gestural hand movement and other hand movement (e.g. scratching, fidgeting). It is also the case that the automatic hand movement annotation captures only the movement phases of a gesture—including preparation and retraction [16 ], and will not pick up any hold phases of gestures, which are known to be interactionally relevant (see e.g. [45 (link)]), particularly in respect to turn-taking.
Analyses were performed in SPSS 28.
Free full text: Click here
