Social Impact Assessment (SIA) facilitates sustainable and equitable consideration of social issues in transport infrastructure planning. However, SIA practitioners face significant constraints in practice. Good SIA is effective in contributing to institutional mechanisms for holding political, bureaucratic and commercial processes accountable for social outcomes. This paper aims to better understand constraints to the assessment and management of social impacts. Drawing on interviews with expert infrastructure professionals involved in projects in Sydney (Australia), Amsterdam (The Netherlands), and internationally, this paper highlights weaknesses in the implementation of SIA, technical engineering-, time- and cost-focused project management practices, and poorly regulated, politically-driven urban governance and planning processes as key constraints. The findings suggest that overcoming those constraints cannot be the sole responsibility of EIA and SIA practitioners and there is more to be considered than practice alone. Instead, this requires an integrated approach across multi-level governance scales and the lifecycle of projects. This paper concludes that for good practice SIA to be realised as intended, systemic adjustments are needed in the planning of urban transport infrastructure, to address constraints and ensure that social impacts are considered in strategic stages and prioritised equally against other project and urban planning issues in decision-making.