This paper considers a risk-constrained infinite-horizon optimal control problem and proposes to solve it in an iterative manner. Each iteration of the algorithm generates a trajectory from the starting point to the target equilibrium state by implementing a distributionally robust risk-constrained model predictive control (MPC) scheme. At each iteration, a set of safe states (that satisfy the risk-constraint with high probability) and a certain number of samples of the uncertainty governing the risk constraint are available. These states and samples are accumulated in previous iterations. The safe states are used as terminal constraint in the MPC scheme and samples are used to construct a set of distributions, termed ambiguity set, such that it contains the underlying distribution of the uncertainty with high probability. The risk-constraint in each iteration is required to hold for all distributions in the ambiguity set. We establish that the trajectories generated by our iterative procedure are feasible, safe, and converge asymptotically to the equilibrium. Simulation example illustrates our results for the case of finding a risk-constrained path for a mobile robot in the presence of an uncertain obstacle.