The effects of COVID-19 on test-retest reliability in a behavioral measure for impulsivity

Predictive power of many behavioral measures relies on high test-retest reliability, whereby a measure yields similar data when repeated measure administration occurs at spaced-out intervals. However, major environmental disruptions between measure administration may impact test-retest reliability. The novel coronavirus (COVID-19) pandemic caused just such a major environmental disruption. We collected impulsivity data via a delay discounting task before, during, and after this environmental disruption. Test-retest reliability was generally statistically significant throughout the study even as delay discounting rates changed in the expected direction between the two experimental groups. Importantly, non-significant correlation coefficients (i.e. poor test-retest reliability) typically occurred immediately after the environmental disruption. Participant’s anecdotal self-reports corroborated COVID-19’s temporary disruptive impact. Although not a planned manipulation, this data provides useful information about whether major environmental disruptions may impact test-retest reliability for events that may not be replicable during a controlled experiment. Social and behavioral scientists attempting behavioral measurement through well-validated measures should be aware of whether large environmental changes can affect measure reliability, and how long such a disruption may last.