We develop a quantitative theory of human capital with heterogeneous agents in order to assess the sources of cross-country income differences. The cross-sectional implications of the theory and U.S. data are used to restrict the parameters of human capital technology. We then assess the model's ability to explain the cross-country data. Our quantitative model generates a total-factor-productivity (TFP) elasticity of output per worker of 2.8. This implies that a factor of 3 difference in TFP is amplified through physical and human capital accumulation to generate a factor of 20 difference in output per worker --- as observed in the data between rich and poor countries. The implied difference in TFP is in the range of estimates from micro studies. The theory suggests that using Mincer returns to measure human capital understates human capital differences across countries by a factor of 2. The cross-country differences in human capital implied by the theory are consistent with evidence from earnings of immigrants in the United States. We also find that TFP has substantial effects on cross-sectional inequality and intergenerational mobility and that public education policies can have important aggregate and distributional implications.