In this work we study the photoinduced signatures of polarons in conjugated polymers and the impact of charge carrier delocalisation on their spectra. The variation of film crystallinity for two prototypical systems - blends of the homopolymer P3HT or the donor-acceptor polymer PCPDTBT with PCBM - allows probing changes of the polaron absorption in the mid infrared spectral region. Increased polaron delocalisation entails a shift of the electronic transition to lower energy in both cases. Also, infrared active vibrations soften due to a higher polymer chain order. Our findings help in providing a more complete understanding of polaron properties in conjugated materials and bring the application of the polaron absorption spectrum as an indicator for the environment on a more thoroughly studied foundation.