Newsweek - National News, World News, Health, Technology, Entertainment and more... | Newsweek.com
SPONSORED BY
Full Post
Posted Friday, January 23, 2009 1:49 PM

An 'Obama Effect' on Blacks' Test Scores?

Sharon Begley

On only the fourth day of his presidency, it’s obviously way too soon to assess whether Barack Obama’s effect on African-Americans will extend beyond providing hope and inspiration. Will he, for instance, goad black students to higher achievement, since he is living proof that working hard can pay off? One intriguing hint of what researchers led by Ray Friedman of the Vanderbilt Owen Graduate School of Management calls the “Obama Effect” suggests that maybe, just maybe, Obama will do more for the scholastic achievement of African-Americans than anything since Brown v. Board of Education.

In a paper under review at the Journal of Experimental Social Psychology, Friedman and colleagues present findings suggesting that Obama might close the black-white gap in scores on standardized tests. That gap reflects, in part, what psychologists call “stereotype threat”. In this now well-established phenomenon, being reminded that you belong to a group that, according to prevailing stereotypes, isn’t good at something causes you to do worse on a test of that something than if you were not so reminded. Similarly, if you are told that you are being assessed on something that stereotypes say your group is not good at (“girls can’t do math”) you do worse than it you’re told the test does not (in this example) detect gender differences. It’s easier to explain by example. When girls who are about to take a math test are reminded of their sex  (basically they just check M or F on a line asking their gender), or when African-Americans about to take a standardized test such as the SAT are reminded of their race, or even when white males take a test that they’re told Asians excel on, they do worse than otherwise. Apparently, students become so anxious about confirming the stereotype that their brains stumble. As the researchers write, “concern about confirming entrenched negative racial stereotypes via poor performance . . . ironically leads to their underperformance on challenging exams.”

So here’s what the new study did. At four different times during 2008 (late August, before the Democratic nominating convention; just after Obama’s acceptance speech; in early October; and right after election day), it asked about 120 college students to take an online test consisting of 20 questions from the Graduate Record Exams (GREs). (Over the four testing periods, 84 black students and 388 white students, matched for education levels, participated.) They were told that the exam was “created by the Massachusetts Aptitude Assessment Center, and is used as a diagnostic tool to assess verbal problem-solving ability”—a ruse meant to activate the stereotype that blacks don’t do as well as whites on aptitude tests. They also had to indicate their race before taking the exam, also known to activate stereotype threat.

Advertisement

The results varied according to when the students took the test. Before the convention and in early October, the performance gap was as wide as ever: white students got a median score of 12.1 compared to blacks’ 8.8 before the convention; the scores were 12.9 and 8.4, respectively, in early October. But just after Obama’s convention speech, and just after election day, “when Obama’s stereotype-defying accomplishments garnered national attention,” as the researchers put it, there was a remarkable effect. Among students who watched Obama’s speech, blacks’ and whites’ scores were statistically equal (10.3 vs. 12.1) after the acceptance speech and 9.8 vs. 11.1 after election day. The difference is considered statistically insignificant--that is, likely due to chance.

Here’s how the researchers see it: “The fact that we found effects with a random sample of American participants, far removed from any direct contact with Obama, attests to the impact that such a powerful role model can have. At the same time, this research provides evidence that real world role models, such as Obama, can trump racial stereotypes only when their success and accomplishments are especially salient. . . . Obama’s impact on Black-Americans’ performance may only occur when his success is highlighted”—that is, the performance gap vanished when his success sparked a media frenzy (acceptance speech, election) but not when it was less visible (pre-convention, early October).

Now the caveats. It is significant that the performance gap vanished in the wake of the acceptance speech only among black students who watched it. Among blacks who did not watch it, scores continued to lag those of whites. That raises the possibility that only some black students will benefit from Obama’s trailblazing. I asked Joshua Aronson of New York University, one of the founders of the research on stereotype threat, what he thought of the study. “They hypothesis [that Obama’s success might eliminate stereotype threat for blacks] makes a lot of sense,” he said. “Studies have shown that even a brief intervention [like watching Obama’s Denver speech] can nullify a stereotype. But the big problem is that . . . participants were not randomly assigned to condition; rather, they self-selected. There looms the strong possibility that the participants who chose to watch Obama's speech and chose to be in the study are the type of students who would be inspired by him and whose test performance would be boosted by thinking about him.” To be sure, there was no such self-selection for the final test condition, after the election; then, everyone knew that Obama had won, and the test-score gap also vanished.

Other experts nevertheless sounded a note of caution similar to Aronson’s, echoing the concern that the Obama Effect may only apply to a particular sub-population of African Americans. Clearly, more experiments need to be done. But the very possibility that Obama’s achievement will pull along an entire generation is intriguing.

You must be a registered user to comment.  Click here to register.  Already a user?  Click here to login.

Member Comments

Posted By: Ben017 (January 26, 2009 at 4:38 AM)

The biggest mistake here: the fact that finding is "statistically nonsignificant" doesn't tell you much at all. If your sample is small, you'll often fail to get statistical significance even if the true gap is large. Lack of evidence of a gap is not the same thing as evidence for lack of a gap. Failure to reject the null hypothesis doesn't prove it, especially in a study with a small sample size. Also, it's my understanding that releasing the results of a study prior to peer review is generally regarded as a red flag.

Also:

1. Note that they give the scores pre-Obama, but not post-Obama. So there was still a gap, but perhaps only at a 90% significance level rather than a 95% level.

2. Sample size of students is tiny, as you noted.

3. The number of questions is very small.

4. Self-selection effects (it's an internet study, not a representative sample).

5. Different times of year may catch students in or out of school, different people who may have heard about the study (including from the experimenters, who may have sent flyers or emails to different places), etc.


Posted By: lsg25 (January 25, 2009 at 4:51 PM)

http://eduoptimists.blogspot.com/2009/01/tuskegee-and-obama-effect.html

By now you've all heard the fascinating news of a study that seems to demonstrate an Obama effect on the black/white gap in test scores. In short, a team led by a Vanderbilt University researcher administered a series of 20 questions (drawn from verbal section of the GRE) shortly before Obama's nomination and again after his acceptance and then again after the election. Black performance on the test improved after Obama's acceptance, and rendered the black/white gap in test performance nonsignificant.

I'm the first to admit the potential for an Obama effect. Every time I hear him speak I think of the power of a role model, and dream of possible studies that could uncover such an effect.

But in this case, I'm not so sure what's being captured is an effect of Obama on the confidence of black students in their academic performance. Here's why:

(1) The students taking the test at each administration were different students. If the same kids took the test repeatedly, obviously we'd expect their scores to increase.

(2) According to the lead researcher, in a personal communique with me, while the pool of potential participants was constructed at time 1, the actual sample at each time was based on volunteers offered a monetary incentive to participate (what size incentive? I don't know).

There are more critical pieces of information missing as well:

(a) Whether the reasons for participation vs. non-participation differed by race, and are correlated with test-taking ability.

and

(b) Whether the rates of participation were similar for both racial groups.

What we do know is that ever since the Tuskegee Study of Untreated Syphilis (TSUS), African Americans are less likely than Whites to volunteer for participation in research. Given the known gaps in achievement, if they knew anything about what the study required they may've also simply lacked the confidence to participate. This is completely understandable. The question is, could it influence the findings in this study? Are there other plausible explanations for the change in test scores observed in the study?

Yes. Let me suggest just a few.

(1) A disproportionate effect of the economy on black's financial status. The study took place during a year of steady decline in the economic standing of many Americans. Is it possible that the money offered for participation wasn't enough to offset the concerns of higher-achieving black students about research (or to offset the opportunity costs associated with participation)? But that by time 2, the money was simply worth more (e.g. more effective as an incentive) and induced greater participation of black students? I'm positing that during the period whites were both less affected by changes in the economy and overall less averse to volunteering to take a test.

(2) An effect of Obama on black's trust in society, including researchers. So at time 1 the black students in the pool are generally more suspicious and only the lower-achievers are affected by the monetary incentive enough to overcome that suspicion and take the test. At time 2, they're feeling more goodwill towards the world, and higher-achieving black students are willing to participate.

(3) Maybe higher-achieving black students, when asked twice to do a study, tend to do it? I don't know if nonrespondents at time 1 were asked again.

These are just three ideas about how sample selection could bias these results. I have many more. What about the gender composition of the samples? ( Black men have lower test scores on average and are generally less likely to participate in studies. )

I want to quantify the good feelings we're all having in the post-Bushie world too. I get the motivation. But I don't think we should get too carried with feel-good stories on studies that have not yet undergone peer review.


Posted By: lollipop1943 (January 25, 2009 at 2:12 PM)

This is to the poster who wrote that the Willie Lynch is a well-known internet HOAX.  I am so glad that I grew up long before the computer, the internet, and Wikipedia was even invented.  I'm old enough to remember when you could actually go to a Library and find a REAL BOOK on an subject you chose.  But America's history has been revised and re-written so much in the last 40 years, till what you find there now is what ever is "Politically correct" at this point in time.  But this is America, and they don't burn books, do they?

 But then I shouldn't be offended, when the Holocaust, where 6 million Jews were slaughtered can now be referred to as just a HOAX..  Tell that to the aging survivors who go around to schools to tell their Personal stories because no one will tell it for them.

 You know what the biggest "RE-EDUCATION" program in this country is going to be?  Trying to convince White people that they are not "superior" to any other human being on this earth.  If God can make a human, why would he make an "Inferior" one?

  Have you ever noticed that any "product" that is not made in America is automatically "inferior"?  And any person not born on American soil is somehow "inferior",  It does not matter if they are White, Brown, Black or Blue.

  It is so sad that soon, books will just be something to make movies out of.  I'm sure "Central Casting" has already selected who will star in the next episode of " Harry Potter."

  AND THE NEWS ISN"T SCRIPtED EITHER.  AND THERE IS AN EASTER BUNNY.