We investigate a mean field optimal control problem obtained in the limit of the optimal control of large particle systems with forcing and terminal data which are not assumed to be convex. We prove that the value function, which is known to be Lipschitz continuous but not of class C1 , in general, without convexity, is actually smooth in an open and dense subset of the space of times and probability measures. As a consequence, we prove a new quantitative propagation of chaos-type result for the optimal solutions of the particle system starting from this open and dense set. (with T. Souganidis)