Classical cooperative game theory is no longer a suitable tool for those situations where the values of coalitions are not known with certainty. We consider a dynamic context where at each point in time the coalitional values are unknown but bounded by a polyhedron. However, the average value of each coalition in the long run is known with certainty. We design "robust" allocation rules for this context, which are allocation rules that keep the coalition excess bounded while guaranteeing each player a certain average allocation (over time). We also present a joint replenishment application to motivate our model.