The assumption that we are typical observers plays a core role in attempts to make multiverse theories empirically testable. A widely shared worry about this assumption is that it suffers from systematic ambiguity concerning the reference class of observers with respect to which typicality is assumed. As a way out, Srednicki and Hartle recommend that we empirically test typicality with respect to different candidate reference classes in analogy to how we test physical theories. Unfortunately, as this paper argues, this idea fails because typicality is not the kind of assumption that can be subjected to empirical tests. As an alternative, a background information constraint on observer reference class choice is suggested according to which the observer reference class should be chosen such that it includes precisely those observers who one could possibly be, given one's assumed background information.