Cosmic microwave background (CMB) constraints on dark matter annihilation are a uniquely powerful tool in the quest to understand the nature of dark matter. Annihilation of dark matter to Standard Model particles between recombination and reionization heats baryons, ionizes neutral hydrogen, and alters the CMB visibility function. Surprisingly, CMB bounds on dark matter annihilation are not expected to improve significantly with the dramatic improvements in sensitivity expected in future cosmological surveys. In this paper, we will present a simple physical description of the origin of the CMB constraints and explain why they are nearly saturated by current observations. The essential feature is that dark matter annihilation primarily affects the ionization fraction which can only increase substantially at times when the universe was neutral. The resulting change to the CMB occurs on large angular scales and leads to a phenomenology similar to that of the optical depth to reionization. We will demonstrate this impact on the CMB both analytically and numerically. Finally, we will discuss the additional impact that changing the ionization fraction has on large scale structure.