Two Data Points are Probably Better than One

College admissions is not an area about which I have a great deal of knowledge, and I think its importance is probably exaggerated by overanxious parents. One thing I do care about, though, is using statistics properly, and this article, concerning a recent College Board study on the efficacy of the SAT, is thus exasperating:

Barmak Nassirian is associate executive director of the American Association of Collegiate Registrars and Admissions Officers and a noted critic of the SAT. He said he views the report as unsubstantial and contends that the SAT did not change enough to make a difference in its predictive quality.

He cites the validity studies to show that high school GPA is about as predictive as the SAT itself. Using the College Board's scale of minus 1 to 1, he notes that high school GPA alone gets a 0.54 while the full SAT gets a 0.53.

This is very strange statistical practice. Mathematically, you cannot do worse using two data points (in this case, high school GPA and SAT scores) to make predictions than you can with one. It makes no sense to decide that because one correlation coefficient is lower than another, you should just toss out what is presumably useful information. How useful it is, of course, depends on correlations. Looking at the College Board report , it appears that using the SAT and GPA together give a correlation of 0.62. (Actually, this is the "adjusted R-squared". The raw results are similar.)

So even if you have the accumulated high school grades representing dozens (hundreds?) of hours of testing, the SAT still adds substantially to the ability to make accurate predictions about college grades. That doesn't mean the SAT has to be used though. Maybe there are better predictors. Maybe colleges should be looking at things other than first year grades. But saying "X predicts better than Y, so let's forget about Y" is silliness.

Share this

How about a post on the value of college?

I think its importance is probably exaggerated by overanxious parents.

Is it worth it to pay 4x more to go to a big name school instead of a state school? I'd be interested to hear your views on that.

Value of College

For what it's worth, I'm having trouble finding reasons for my 18- and 16-yo sons to go to college. Even scanning through the community college catalog, it's hard to find relevant courses--maybe math, engineering, CAD, and welding. And for each of these, I'm not sure of the value of accreditation as opposed to just having the knowledge.

For better or worse, their lives have been homeschooling -> self-study -> self-employment. I'm not sure if this is the way of the future, or just some genetic thing.

Two videos on the topic:

Recent stuff from Michael Wesch on how benefits could be better:

Gary North on how costs are too high:

I'll try to write something

I'll try to write something more detailed on it later, but my quick and dirty answer is no, it isn't. For me, the best study on this question was Dale and Krueger's paper (publically accessible version here). Basically, they took students accepted and rejected by similar sets of institutions, and they found the ones who chose to go to more selective ones did not earn more money. But other people have found different results, so I don't think it's a settled question yet.

BIG caveat to the paper: If you look at what institutions they look at, most are pretty elite. So I think it supports the conclusion that it doesn't matter if you choose UNC over Penn, but not that you could go to Northwestern Podunk State and be just as well off financially.

Is it worth it to pay 4x

Is it worth it to pay 4x more to go to a big name school instead of a state school?

The answer, I think, is: it depends. Namely, it depends on what you plan on doing after graduation. If you plan on entering a very competitive field, and think you have what it takes to win one of the top jobs without having to work for a few years first to prove yourself, then it worth paying to go to the best school that is willing to accept you. I know this is the case with law schools and students wanting to work at high-end corporate law firms immediately following graduation. It's also probably true for MBA programs, assuming the students plan to work for an existing firm and not go the entrepreneurial route. You would know more about whether it's also true for medical school, but I would be very surprised if it wasn't true for things like surgery.

It's probably most true for people going into academia, especially law professors but probably also economics. But academia is all about reputation and fame, and less sensitive to profit-maximizing market forces. The more prestige matters in a career, the more important prestigious indicators such as school reputation (as measured by expense) becomes.

Oh, I think it's definitely

Oh, I think it's definitely true in economics. Harvard and MIT dominate the profession to an astonishing (and, to my mind, unhealthy) degree, and Harvard at least is well known for favoring applicants from their own program. But the degree of Cambridge-centeredness is less so than in the past. See tables 1 and 2 of this ranking based on placement. The numerical scores for each department are much more closely bunched looking at people awarded PhDs after 1990 than at the entire sample. And it's not like you can't make it out of other schools. Probably the star from this past job market came from my top-10-but-not-top-5 school, and frankly I suspect our placement record was the best this year.

School Choice

It depends on what you want.

"Like low SAT scores, incompetent professors, rock-bottom admissions standards, unbridled alcohol and drug consumption, rampant criminal activity, and dubious alumni."


Good Schools

unbridled alcohol and drug

unbridled alcohol and drug consumption, rampant criminal activity, and dubious alumni

I went to the wrong school; we had none of this. Well, maybe some dubious alumni.

Wow. That article was

Wow. That article was fantastic:

On the verge of extinction in the early 1990s, the university was rescued from bankruptcy by an arm of Reverend Sun Myung Moon's Unification Church, which offered up a multimillion-dollar miracle. The Moonies' endowment eventually earned the former head of the church a position as university president, and a head scratch from the academic world. Alumni were horrified to learn that a cult long accused of fraud, high-pressure recruitment tactics, and wrenching troubled kids from their parents had effectively mounted a takeover of their alma mater.

If X predicts better than Y,

If X predicts better than Y, it might make sense to drop Y to avoid over fitting. If you look at a lot of variables, income of the parents, height, weight, degree of myopia, age, number of siblings, etc you will find correlation between all of those will probably be correlated to what you are measuring. Some correlations will be spurious, but let's say you are very rigorous in eliminating those, by keeping only high correlations and pairs where a causal link is obvious.

Even then, you will find an inflated prediction if you try to predict using all the available data, because you will have spurious correlations in the correlation matrix of the regression variables.

The more predictive variables you use, the more likely you are to be overconfident in your accuracy, and the more likely you are to produce a result which is simply wrong.

Thus in many cases, it's better to use less, more significant data points than to try and combine everything.

Interesting. If you had been

Interesting. If you had been my professor for econometrics I probably wouldn't have had to drop it twice.

Are you familiar with Deidre McCloskey and her "oomph" criticisms of statistical significance?

Not at all, but since you

Not at all, but since you mention it, I went briefly through this... from this I don't really get the gist of her criticism.

College is a scam

In the 50's a high school grad was qualified for most of the entry level positions or an apprenticeship. These days a kid needs 2 years of college to get a high school education. Half the freshman class at the University of Washington (8.8+ or so GPA) need remedial math classes.

BS degrees are a dime a dozen. Another few years and one will need 2 years of college to drive a garbage truck. Skilled workers, machinists, electricians . . . are scarce. Half the kids getting college loans will never break even unless they are going for a govt job. The average kid will be dollars ahead if he gets an apprenticeship and a union job right after high school.

The average kid will be

The average kid will be dollars ahead if he gets an apprenticeship and a union job right after high school

Yeah, that or mugging people, union jobs are safer though.