This paper examines how family background affects the influence of childhood ability on educational attainment and future earnings. In it, we find that ability measured by various IQ tests taken at age 12 predicts future success for males about equally well. And that his returns to ability based on his earnings at age 43 and 53 are rather stable. For females, however, explanatory power and returns of various IQ scores are falling over time. We also find that cognitive ability is rewarded differently across families. The observed differences are consistent with the idea that accessibility to family resources influences the educational attainment of children, and therefore future earnings. We find that fortunate families are better equipped to provide their low ability scoring children with compensating human capital to overcome early learning deficiencies. And we find that imperfect credit markets especially hit smart boys coming from poor families where lacking resources restrict their educational career.