Investigating the validity of picture-based assessments across cultures and contexts

Abstract

Many childhood assessments rely on picture stimuli, but children in diverse early environments possess varying amounts of experience with pictures. Two preregistered experiments, conducted in 2022-2023, investigated whether picture assessments are valid across diverse contexts. Low-to-middle-income children (n = 192, 2-7 years, 85 females, all Black) in their first month of formal schooling in Mombasa County, Kenya, an early environment with relatively few pictures, performed more accurately on an object vocabulary task than a picture vocabulary task (beta = 0.07, p < .001; Experiment 1). Middle-to-high-income children (n = 96, 2-3 years, 52 females, predominantly White and Asian) in the San Francisco Bay Area, an early environment with relatively more pictures, performed similarly on object and picture vocabulary tasks (beta = 0.02, p = .60; Experiment 2). Consequently, these results tentatively suggest that assessments involving pictures may underestimate children's capacities in some contexts. To accurately measure developing capacities in children from diverse backgrounds, it is critical that assessment tools are appropriately adapted to environmental contexts.