Tracking the truth by selecting good data: Coherence measures and data selection

In formal epistemology, a variety of probability-based coherence measures have been proposed that provide a quantitative formal representation of the coherence of a set of information pieces. While research has long focused on whether coherence measures are truth-conducive, the truth-conduciveness of coherence measures has so far been evaluated in static settings only: Coherence provides assessments about the truth of incoming information, but does not actively guide decisions to believe or discard pieces of information. In this paper, we propose to assess the truth-conduciveness of coherence measures with respect to their ability to lead agents to select true information and form correct beliefs in a dynamic iterative setting. At every time step, an agent receives a number of noisy signals about the actual truth values of a finite set of atomic propositional variables. The agent uses a coherence measure to decide which signals to trust and which to discard. By repeatedly picking signals that maximise the coherence of the propositions they currently believe to be true, the agent tries to select truthful signals and learn the correct truth-value assignment for the atomic variables. The contribution of this paper is three-fold. First, we propose a computational model to assess the truth-tracking abilities of different coherence measures. Second, using computational simulations, we compare a number of widely discussed coherence measures from the novel standpoint of our iterated data-collection setting: We show that, when signals are not too noisy, agents who employ the Glass-Olsson relative overlap measure outperform agents employing all other tested measures, and that all measures become progressively worse at leading agents towards the truth as signals degrade. Finally, we discuss how coherence affects the emergence of different dynamics and attitudes in belief revision.