Newsweek - National News, World News, Health, Technology, Entertainment and more... | Newsweek.com

Nurture Shock

SPONSORED BY
Full Post
Posted Friday, September 18, 2009 9:09 AM

In Defense of the SAT

Po Bronson
One of the most popular ideas of our time is the notion that in judging a young person’s future success, we’ve become imbalanced, giving too much credence to whether a child has learned the stuff of textbooks, and too little value to whether that child has learned the stuff of real life.

The latter is a whole constellation of behaviors and skills, from creativity to emotional-intelligence to self-discipline to practical judgment. In this modern paradigm, the elements of real life success are characterized as highly generalizable, useful everywhere from the urban street corner to the boardroom. Meanwhile, the elements that go into book learning are characterized as being narrowly applicable, useful only for getting into college, at which point the other factors take over.

No matter who is making this argument–whether it’s Daniel Goleman, Dan Pink, Robert Sternberg, Malcolm Gladwell, Thomas Stanley, or some college president–it always stands on a few key bricks. One of those bricks is that the SAT doesn’t predict much of anything.

It’s commonly said that the SAT, taken in a senior year of high school, has only about a 40% correlation with a student’s freshman year college GPA. If it’s that bad at predicting how well a kid does in college, just one year later, then how could it predict longer-term outcomes in life, when other factors become increasingly important? The SAT is designed, specifically, to screen for college success–if it doesn’t accomplish what it’s built for, then surely something else (that’s not being tested) accounts for real success, in college and in life.

Using this argument, the door is opened for all these other variables to be postulated as the new basis for success.

I’ve always had a skeptical feeling about the 40% correlation statistic, and so I’ve never relied on it or used it in print. There are two self-selection problems that make it really hard to control the data. First, high schoolers of diverging abilities apply to different schools–the strongest students apply to one tier of colleges, and the average students apply to a less ambitious tier, with some overlap. Second, once students get to a college, they enroll in classes they believe they can do well in. Many of the strongest students try their hand at Organic Chemistry, while more of the less-confident students take Marketing 101. At each of these colleges and courses, students might average a B grade, but the degree of difficulty in achieving that B is not comparable.

Many scholars have attempted to control for these issues, looking at data from a single college or a single required course that all freshman have to take, and their work has suggested the 40% correlation is a significant underestimate. I’ve long wondered what would happen if an economist really took on this massive mathematical mess, on a large scale, harvesting data from a wide selection of universities.

Finally this has been done, by Christopher Berry of Wayne State University and Paul Sackett of the University of Minnesota. They pulled 5.1 million grades, from 167,000 students, spread out over 41 colleges. They also got the students’ SAT scores from the College Board, as well as the list of schools each student asked the College Board to send their SAT scores to, an indicator of which colleges they applied to. By isolating the overlaps–where students had applied to the same colleges, and taken the same courses at the same time with the same instructor–they extracted a genuine apples-to-apples subset of data.

It turns out that an SAT score is a far better predictor than everyone has said. When properly accounting for the self-selection bias, SAT scores correlate with college GPA around 67%. In the social sciences, that’s considered a great predictor.

My point isn’t that other life variables don’t matter. Some of the success factors that have been touted are certainly additive to what’s tested by the SAT. It’s still worthwhile to explore why people succeed, both at school and in real life. But we may not be imbalanced; our valuing of differing abilities may be right on target. Meanwhile, the argument all these alternative success-factors are built on needs repair.

Advertisement
Tag(s):
You must be a registered user to comment.  Click here to register.  Already a user?  Click here to login.

Member Comments

Posted By: Mark Coatney (September 21, 2009 at 10:49 AM)

While success can be based on many factors, some fields of study use the SAT as a measure of 'keeping up with the Joneses'. Engineering schools want high Math scores (to follow along with profs who write a dozen equations on the board); likewise, journalism schools want high verbal scores.

Because grades are subjective, the SAT is a broader indicator. Due to the absence of a national curriculum, we can think of the SAT as a unifier for a reasonable body of knowledge for high schoolers.

As a Math peak performance coach, I have found that studying for the SAT can be challenging and entertaining while promoting brain fitness at any age. Although I graduated from college in 19XX, this October 10th I will take the SAT to gain perspective, to have fun, and to boost mental fitness. Click here for the SAT Question of the Day http://apps.collegeboard.com/qotd/question.do

Robin Schwartz

www.mathconfidence.com


Posted By: Mark Coatney (September 20, 2009 at 2:12 PM)

It's been many years since I took the SAT (the first time being in the 9th grade), and one thing stuck with me about the exam. I was always told that the SAT reflects my ability to learn, and in how large of a capacity. If there's a 40% to 67% (an unreliable number for this source) correlation between the test and success, why are we still pushing high school students? I think the SAT has become one more layer of BS in which a company creates or enhances a myth to make money. We really should re-evaluate this test's use.

To use the College Board as a reliable source for a study in an article such as this is flimsy journalism at best as it is in College Board's interest to promote college testing as a means of providing "useful" content to potential users. Simply put, promoting these standardized college exams helps College Board remain a successful business. And a business it is. If you go to it's website, you'll see that it is a .com not a .org and is a "not-for-profit" business not a "non-profit" business which still means that it looks to make profit for use in growing and securing it's long term interests.

See this May article in Mother Jones:

http://www.motherjones.com/mojo/2009/05/sat-and-nonprofit-revenue-machines

To write an article in support of the SAT, it would have been prudent to offer more information about this Wayne State/Minnesota State study such as how much of the demographic spectrum was used, what areas of the country were used and to what degree, and most importantly.... when did this study take place. With major educational legislation being enacted in the No Child Left Behind Act now having a huge impact on the those taking standardized college tests, it is of vital importance to know when this study is from and over how long of a period was the data being used taken from.

It would have also been prudent to not rely on the company that is THE SAT.


Posted By: Anonymity (September 19, 2009 at 9:53 PM)

I thought Bronson's article was not only poorly argued but also dangerous if misinterpreted. As one who has taken the SAT (and has done pretty well), I still think that the entire idea fosters incorrect and prejudicial ideas of intelligence. There are just too many factors that hinder proper analysis--which is why I think that the SAT and all standardized tests should be abolished.

First, the SAT does not assess all the subjects taught in an average curriculum. Rather, it does the exact opposite. The SAT only limits the material to mathematics, reading, and writing. This completely ignores subjects like arts, policy matters, social science, and even science (which is the highest and fastest growing field).

Second, the SAT is classist. Even though the SAT tries to provide resources for the impoverished, it gives an advantage to the affluent. (1) SAT prep classes are often too expensive and cater to the wealthy. (2) Terms in the SAT like "Bollard" (which means a yacht post) are more likely to be known by those who own a yacht. I would like to see ebonics on the test.

Third, the SAT preconceives that the only way to learn is through reading it in a text book. It completely ignores visual, auditory, and kinesthetic learners. It is an atrocity that an deserving student can be rejected from most colleges because he/she doesn't conform to SAT teaching.

Fourth, the empirical data of 66% only refers to freshman, core class GPA (math, english, etc.) College, however, is a multi-faceted institution with a wide variety of classes. I bet if you took the same study with art, language, or dance, those visual, auditory, and kinesthetic learners would be properly represented.

Now I'm not saying that the SAT determines one's admission to college, but many universities reject one application upon seeing their SAT score. In addition, it leads  a student to underestimates themselves and no one should do that. Thus I applaud the colleges that disregard the tests.

A defense of the SAT is a defense of a prejudicial, classist, and corrupt system.