I propose a quantitative theory for the dynamics of wealth inequality based on households' heterogeneous exposure to aggregate risk in asset returns. I develop an optimal portfolio choice model building on evidence that housing is a necessary good which replicates the main features of portfolio shares along the wealth distribution: as households get wealthier they shift their portfolios away from safe assets, first towards housing, and then towards equity. Since households in different parts of the wealth distribution are exposed to different sources of aggregate risk, the model has strong implications for the evolution of inequality. In particular, I show that temporary shocks in equity returns have large and persistent effects on top wealth inequality and that the observed rise in U.S. top wealth shares since the 1980s was mostly due to abnormal equity returns. Finally I also show that, to the extent that these changes were transitory, inequality is likely to revert back to lower levels.