Dimerization free energies are fundamental quantities that describe the strength of interaction of different molecules. Obtaining accurate experimental values for small molecules and disentangling the conformations that contribute most to the binding can be extremely difficult, due to the size of the systems and the small energy differences. In many cases, one has to resort to computational methods to calculate such properties. In this work, we used molecular dynamics simulations in conjunction with metadynamics to calculate the free energy of dimerization of small aromatic rings, and compared three models from popular online servers for atomistic force fields, namely G54a7, CHARMM36 and OPLS. We show that, regardless of the force field, the profiles for the dimerization free energy of these compounds are very similar. However, significant care needs to be taken when studying larger molecules, since the deviations from the trends increase with the size of the molecules, resulting in force field dependent preferred stacking modes; for example, in the cases of pyrene and tetracene. Our results provide a useful background study for using topology builders to model systems which rely on stacking of aromatic moieties, and are relevant in areas ranging from drug design to supramolecular assembly.