Stay-Switch Analysis of Two-Step Task Behavior

The one-trial-back stay-switch analysis is the most widely used method for characterizing behavior on the two-step task (Daw et al., 2011 (link); Akam, Costa, & Dayan, 2015 (link); Wunderlich et al., 2012 (link)). This method quantifies the tendency of a participant to repeat the choice made on the last trial or switch to the other choice, as a function of the outcome and transition on the previous trial. We considered four possible outcomes: Common-Rewarded (CR), Rare-Rewarded (RR), Common-Unrewarded (CU), and Rare-Unrewarded (RU). Model-based and model-free indices were computed from the stay probabilities following each outcome according to:
$\begin{array}{rcl}\text{MF}& =& (P(\text{stay}|CR)+P(\text{stay}|RR))-(P(\text{stay}|CU)\\ & & +P(\text{stay}|RU)),\end{array}$ $\begin{array}{rcl}\text{MB}& =& (P(\text{stay}|CR)+P(\text{stay}|RU))-(P(\text{stay}|CU)\\ & & +P(\text{stay}|RR))\mathrm{.}\end{array}$
We also examined whether hunger modulated other measures of simple reinforcement learning. We found no effect of hunger on changes in model-free control for second-stage choices or for action-specific, stimulus-irrelevant choices at the first-stage (Fig. S1).
Statistical analyses were implemented in MATLAB and SPSS (IBM Corp. Released 2019. IBM SPSS statistics for Windows, Version 26.0. Armonk, NY: IBM Corp.). We report the effect size with Cohen’s d and ${\eta }_p^2$ and report the 95% confidence interval (CI) of the difference between groups.