Recent scientific investigations suggest that people automatically mimic each other’s pupil sizes during interaction. However, instead of being a social mimicry effect, it could also be the result of brightness perception. When observers look at individuals with dilated pupils, little of the brighter iris is visible, leading to the perception of a relatively low-illuminated eye region. In the current study we tested whether pupil mimicry remains present when pupils and irises are equalized for luminance values across pupil sizes. We tested several stimulus sets, including faces with static pupils that varied in size across images and dynamic pupils that changed in size over time in videos. Results showed that for traditional, not-luminance-equalized videos, participants’ pupil sizes adapted to the observed pupils, showing a pattern that is roughly in line with pupil mimicry. However, no such pupil response in line with mimicry was seen for static images (regardless of whether they were equalized for luminance) nor for luminance-equalized videos. These findings suggest that only salient, dynamic stimuli attract enough attention to the luminance in the eye region to evoke a pupillary response. However, although such responses suggest pupil mimicry, the underlying factor is the change in brightness within the eye as a function of pupil size rather than social mimicry.