There is a need for simple and effective ways to screen for visual field defects (VFD). Watching a movie is a simple task most humans are familiar with. Therefore we assessed whether it is possible to detect and reconstruct visual field defects based on free viewing eye movements, recorded while watching movie clips. Participants watched 90 movie clips of one minute, with and without simulated visual field defects (sVFD), while their eye movements were tracked. We simulated homonymous hemianopia (HH) (left and right sided) and glaucoma (small nasal arc, large nasal arc, and tunnel vision). We generated fixation density maps of the visual field and trained a linear support vector machine to predict the viewing conditions of each trial of each participant based on these maps. To reconstruct the visual field defect, we computed "viewing priority" maps and maps of differences in fixation density of the visual field of each participant. We were able to classify the simulated visual field condition with more than 85% accuracy. In simulated HH, the viewing priority distribution over the visual field indicated the location of the sVFD in the simulated HH condition. In simulated glaucoma the difference in fixation density to the control condition indicated the location of the sVFD. It is feasible to use natural viewing behavior to screen for and reconstruct (simulated) visual field defects. Movie clip viewing in combination with eye tracking may thus provide an alternative to or supplement standard automated perimetry, in particular in patients who cannot perform the latter technique.