Phononic materials are artificial composites with unprecedented abilities to control acoustic waves in solids. Their performance is mainly governed by their architecture, determining frequency ranges in which wave propagation is inhibited. However, the dynamics of phononic materials also depends on the mechanical and material properties of their constituents. In the case of viscoelastic constituents, such as most polymers, it is challenging to correctly predict the actual dynamic behavior of real phononic structures. Existing studies on this topic either lack experimental evidence or are limited to specific materials and architectures in restricted frequency ranges. A general framework is developed and employed to characterize the dynamics of polymer phononic materials with different architectures made of both thermoset and thermoplastic polymers, presenting qualitatively different viscoelastic behaviors. Through a comparison of experimental results with numerical predictions, the reliability of commonly used elastic and viscoelastic material models is evaluated in broad frequency ranges. Correlations between viscous effects and the two main band-gap formation mechanisms in phononic materials are revealed, and experimentally verified guidelines on how to correctly predict their dissipative response are proposed in a computationally efficient way. Overall, this work provides comprehensive guidelines for the extension of phononics modeling to applications involving dissipative viscoelastic materials.