Stimulus norming: It is too soon to close down brick-and-mortar labs

Psycholinguists grapple with an ever-increasing list of control variables, in addition to any that are of theoretical interest. Some variables are subjective constructs like familiarity, concreteness, and semantic or affective connotations. Historically researchers approached these by having participants come to a laboratory and provide ratings for each stimulus, but the use of the Internet in data collection has increased in recent years and is likely to continue doing so. In the context of stimulus norms, the equivalence of lab-based and Internet methodologies has not been extensively examined. We discuss some of the pros and cons of online stimulus norming and touch on several issues to consider. We also highlight some important differences between norms obtained online and those obtained in-person.