The biggest mistake here: the fact that finding is "statistically nonsignificant" doesn't tell you much at all. If your sample is small, you'll often fail to get statistical significance even if the true gap is large. Lack of evidence of a gap is not the same thing as evidence for lack of a gap. Failure to reject the null hypothesis doesn't prove it, especially in a study with a small sample size. Also, it's my understanding that releasing the results of a study prior to peer review is generally regarded as a red flag.

Also:

1. Note that they give the scores pre-Obama, but not post-Obama. So there was still a gap, but perhaps only at a 90% significance level rather than a 95% level.

2. Sample size of students is tiny, as you noted.

3. The number of questions is very small.

4. Self-selection effects (it's an internet study, not a representative sample).

5. Different times of year may catch students in or out of school, different people who may have heard about the study (including from the experimenters, who may have sent flyers or emails to different places), etc.

http://eduoptimists.blogspot.com/2009/01/tuskegee-and-obama-effect.html

By now you've all heard the fascinating news of a study that seems to demonstrate an Obama effect on the black/white gap in test scores. In short, a team led by a Vanderbilt University researcher administered a series of 20 questions (drawn from verbal section of the GRE) shortly before Obama's nomination and again after his acceptance and then again after the election. Black performance on the test improved after Obama's acceptance, and rendered the black/white gap in test performance nonsignificant.

I'm the first to admit the potential for an Obama effect. Every time I hear him speak I think of the power of a role model, and dream of possible studies that could uncover such an effect.

But in this case, I'm not so sure what's being captured is an effect of Obama on the confidence of black students in their academic performance. Here's why:

(1) The students taking the test at each administration were different students. If the same kids took the test repeatedly, obviously we'd expect their scores to increase.

(2) According to the lead researcher, in a personal communique with me, while the pool of potential participants was constructed at time 1, the actual sample at each time was based on volunteers offered a monetary incentive to participate (what size incentive? I don't know).

There are more critical pieces of information missing as well:

(a) Whether the reasons for participation vs. non-participation differed by race, and are correlated with test-taking ability.

and

(b) Whether the rates of participation were similar for both racial groups.

What we do know is that ever since the Tuskegee Study of Untreated Syphilis (TSUS), African Americans are less likely than Whites to volunteer for participation in research. Given the known gaps in achievement, if they knew anything about what the study required they may've also simply lacked the confidence to participate. This is completely understandable. The question is, could it influence the findings in this study? Are there other plausible explanations for the change in test scores observed in the study?

Yes. Let me suggest just a few.

(1) A disproportionate effect of the economy on black's financial status. The study took place during a year of steady decline in the economic standing of many Americans. Is it possible that the money offered for participation wasn't enough to offset the concerns of higher-achieving black students about research (or to offset the opportunity costs associated with participation)? But that by time 2, the money was simply worth more (e.g. more effective as an incentive) and induced greater participation of black students? I'm positing that during the period whites were both less affected by changes in the economy and overall less averse to volunteering to take a test.

(2) An effect of Obama on black's trust in society, including researchers. So at time 1 the black students in the pool are generally more suspicious and only the lower-achievers are affected by the monetary incentive enough to overcome that suspicion and take the test. At time 2, they're feeling more goodwill towards the world, and higher-achieving black students are willing to participate.

(3) Maybe higher-achieving black students, when asked twice to do a study, tend to do it? I don't know if nonrespondents at time 1 were asked again.

These are just three ideas about how sample selection could bias these results. I have many more. What about the gender composition of the samples? ( Black men have lower test scores on average and are generally less likely to participate in studies. )

I want to quantify the good feelings we're all having in the post-Bushie world too. I get the motivation. But I don't think we should get too carried with feel-good stories on studies that have not yet undergone peer review.

This is to the poster who wrote that the Willie Lynch is a well-known internet HOAX.  I am so glad that I grew up long before the computer, the internet, and Wikipedia was even invented.  I'm old enough to remember when you could actually go to a Library and find a REAL BOOK on an subject you chose.  But America's history has been revised and re-written so much in the last 40 years, till what you find there now is what ever is "Politically correct" at this point in time.  But this is America, and they don't burn books, do they?

 But then I shouldn't be offended, when the Holocaust, where 6 million Jews were slaughtered can now be referred to as just a HOAX..  Tell that to the aging survivors who go around to schools to tell their Personal stories because no one will tell it for them.

 You know what the biggest "RE-EDUCATION" program in this country is going to be?  Trying to convince White people that they are not "superior" to any other human being on this earth.  If God can make a human, why would he make an "Inferior" one?

  Have you ever noticed that any "product" that is not made in America is automatically "inferior"?  And any person not born on American soil is somehow "inferior",  It does not matter if they are White, Brown, Black or Blue.

  It is so sad that soon, books will just be something to make movies out of.  I'm sure "Central Casting" has already selected who will star in the next episode of " Harry Potter."

  AND THE NEWS ISN"T SCRIPtED EITHER.  AND THERE IS AN EASTER BUNNY.

GMAB

This isn't rocket science, people. 50Cent, BowWow, etc have inspired a very sad number of blacks to become gun-carrying drug dealers.  Just like Brittany and Madonna inspired a lot of white girls to be 'tween sluts. IMHO, neither race has an excess of positive 'role models'. The Enquirer can't sell papers based on people being honest and doing good things.

So if Obama can inspire even one child that would have gone the way of 50Cent, good for him. I don't think it's as simple as one role model, but if one child rethinks their lives and decides that they want to be more, I'm all for it.  

The media loves to glorify the 50Cents and Brittany's of the world. It sells papers. Kids are constantly bombarded with 50Cents self-proclaimed 'gangsta ***' persona and Paris Hilton's "love me just because I'm pretty and rich' BS. So however Obama's presidency turns out, hopefully he will continue to be a positive influence in a society that desperately needs positive role models.

Being the president, it's not like they (the media) can push him to the back of the paper for 50Cent or Brittany's front page antics. So he will get coverage, and he will get at least marginally glorified.

Along with my prior post, I would like to reiterate that it is the content of the school curriuclum and the perceptions of the teachers about their students' ability (and vice-versa) that stunt the academic purpose of many African American students and Latino students.

Additionally, many of these students are swallowed up in Special Education enrollments that are not student -centered but centered on a school culture of hidden curriculum and agenda.  Advanced placement sequential progression in classes during high school is not the route most African American and Latino students are given.  The instruction and skill levels that one gets from a bridge class, to an honors class, prepares a student to be college bound and two or more advanced placement courses will have them on their way.  But that consistently does not happen in our schools.

So, it is not important whether or not Black students reflect upon the success of Barack Obama; what is important is that they get a fair and balanced depiction of who they are within the content of what is taught in schools, and the same access and programming into classes that will make them viable in a post-secondary setting.  The students will engage with their teachers and classmates more forthrightly and with proper intent when they feel they are given value both in thought, and deed.

The article neglects to mention if the people sampled were forced to watch or not watch the speeches.  I think this matters because it seems reasonable to expect people who actually care about things in general would watch a speech by someone who may have an impact on their future.  These people are probably more likely to study for exams as well.  People who are aspiring losers probably did not take the time to watch a speech of any type, thereby dragging that group of peoples scores down.  This article is just another attempt by the media to attribute greatness to a man who has only proven that he is an inspirational speaker and has an unlimited amount of "giving someone the benefit of the doubt".

This artical infuriates me to no end.  I feel sorry for Obama because the MEDIA has handed him an enormous pair of shoes  that he must somehow be able to fill.  He is a man...he is not a God and it would do the MEDIA and everyone else good to remember that.  The MEDIA is the true enemy here not the white man.  I am tired of the excuses that people make for their 'failures' in life.  WE ALL HAVE TO POWER TO SUCCEED REGARDLESS OF THE COLOR OF OUR SKIN!!!!  Obama is an example of that as well as many other black people.  The MEDIA are the ones creating this racial divide and to be honest...I am sick of it!  This constant 'in your face' about black people rights is far out played.  Quit living in the past...look at where we have come as a national over the past fifty years.  I am so proud of this country because we can live in unity...however if the MEDIA continues playing this race card it is only going to break down the great strides that we have made over the years.  I think we the people need to stand together and turn our backs to the MEDIA.  Maybe if we stop listening then they will stop reporting.

One more thing...my fiance is a white man from South Africa.  Once he becomes an American citizen that will make him an African-American.  You want to know why?  Because he was BORN in Africa.  Quit using the term African-American.  If you were born in America that makes you an American.  

If it takes Obama's celebrity and brillance to motivate students to achieve, then I aint complaining.

The Willie Lynch speak mentioned by a user in comments is a well - known Internet hoax. Don't perpertrate it.

http://en.wikipedia.org/wiki/William_Lynch_Speech

Where have you people been living?  This has been going on for centuries.  It's called "Prior Conditioning", "Power of Suggestion", " Subliminal Imagining",etc. Or what ever Modern Science want to call it.  But the GRANDADDY  of them all, that relates to Black people is the Willie Lynch Theory.  This was introduced to the American slave owners in 1723.

  If you have not read it, go to www.thetalkingdrum.com  and click on "The Making of a Slave."  And for those who say that the hundreds of years that Blacks lived under slavery should have no effect upon them now, I say "Balderdash."  If you grew up in the North and then move to the South you will see the PROFOUND effect has had on those who never left.

  It's sort of like the elephants that were trained to stay inside a roped off area.  After the rope was no longer put in place, the elephantsremained inside that area as if they couldn't get out.

  Ask Paul and Jan Croutch about "Conditioning."   You ever notice that whenever they go on their missions to Africa, they always take a truck full of dolls that look just like Jan Croutch?

 Pretty soon all of us older people will have died off, and people will start thing that this is something BRAND NEW.  There is nothing NEW UNDER THE SUN.  It's just going back around.

Yeah, and maybe President Obama will end the conflict in the Middle East next week, eradicate poverty next month, and usher in world peace by the end of the year.

These false expectations of what President Obama can/will do only sets people up for cynicism and mistrust in the long run and weaken his chances of success.

The people I know (regardless of ethnicity) who have made positive strides forward in academics, business, or relationships have done so with the encouragement and support of people around them.

While President Obama may serve as an inspiring figure, it will be the people walking alongside those wanting to change, through the difficult days of unlearning bad habits and developing positive habits, who deserve the credit and praise.

While inspiration may initiate movement, it is perspiration that produces the transformation.

Wisdom and experience warn me to be cautions in assigning too much value to one person.  

What the author stipulates is a perspective that is insightful.  The "Obama Effect" is the general aura of auspaciousness that is promoted towards African American icons in multicultural studies.  They are larger than life, and nearly impossible to emulate.  These individuals, extraordinary in any measure, or the persons extolled in conventional celebrations of black culture.  President Obama is now a new icon, and his power to impress is limited also, because what comes to bear, all too soon, is that most African Americans are not viewed by Anglo Americans with the same perceptions that most have towards a Martin Luther KIng, et alia.

To combat stereotype threat and cultivate the buy-in nedded to participate in the great social change that potentially can become a reality, the content of school curriculum must be modified to speak to the antithesis of the concepts given in class instruction.  If an English teacher is going to use Mark Twain in teaching 19th century realism, they must deconstruct ethnic notions and discuss them in the classroom.  Teachers need to add resource material to class instruction where minority student representation in the texts can become slanted.  Censorhip is not what is being asked for here, but transparency and an honest attempt at inclusion.

Last year, after decades of relative parity in high school graduation rates for White and Black students (approximately 78%), last spring those numbers for Black students dropped 45%.  Other studies have shown that even when the grad rates were even, only half of the Blacks who graduated from high school would be accepted into college.  So, yes, there is a great concern.  And it is not hyperbole.

There is an opportunity to engage a whole generation, maybe two of African American youth to do what many Whites have been begging for them to do for ages: become engaged and respect getting an education.  With Obama, there is a signifier, someone who has modeled the dream on a large canvas (though Black professionals have been doing the same thing for years).  What a shame it would be let this opportunity get derailed because the dominant culture now believes they have fulfilled paying a debt, instead of seeing that a door has just been opened to truly freeing the minds of Black youth.

I'm not sure about this particular study, but gender studies have indeed been done, and in areas where one might expect stereotype threat to show up (such as when women are reminded just before a test that men usually do better on this math/science test), stereotype threat does prevail.  Similarly, women not reminded of their gender on math/science tests do better than women who are so reminded.

These comments are incredibly depressing. This is an interesting, not particularly surprising, social psychological result with some hope for small but real positive trends in our society. And nitwits here are acting as though they've been thrust into a race war.

I wonder what role gender has in this mix? Do women [of either race] score lower because of the lack of female role models at the highest levels? And did the study consider gender in its methodology?

Stereotype Threat has been an accepted, and proven, aspect of social psychology for a long time.  The public never seems to accept principles of Social Psychology until it stares them in the face, such as with the Bay of Pigs or Challenger explosion, each of which demonstrated with horror the dangers of "group think".  This is the only subject I teach in the university; it's real, folks!

and the race war begins...

We are really looking forward to another worthless piece of journalism that appears to be a complete suspension of disbelief.  The Obama Effect?  Pu-lease.  That is a complete stretch and another conjured piece of propaganda to create more divisiveness in the country.  I thought Obama's big message and platform was that he was going to 'bring people together'?  I knew he was slicing baloney, too bad others were in a political stupor and played; completely and utterly played.  I wouldn't care if this guy was purple with green polka dots, it's all about the issues at hand.  The President is a PUBLIC SERVANT, not some kind of king where he can pick and choose whom he wants to represent and help live in peace.  We just had 8 long years of that 'me so bad and I can do as I want' nonsense.  We traded one cowboy for another?  

I love this!  Is it about race?  YES!  So, let's ban the NAACP, the United Negro College Fund, the Black Congressional Caucus and any other organization that thrives on the very fact of RACE as it's agenda!  I keep hearing the phrase "it's time for change"...then IT'S TIME FOR CHANGE!

What about the half white/half black kids?  Will they benefit from 'Bama's rise to power?

AND STOP USING THE TITLE "AFRICAN-AMERICAN" AS AFRICA IS A CONTINENT, NOT A COUNTRY.  'BAMA IS A KENYAN-AMERICAN, NOT AN AFRICAN-AMERICAN!  GET OVER IT PEOPLE...IT'S TIME FOR CHANGE!  'BAMA IS NOT BLACK, HE IS HALF WHITE!

To Obama's Hollywood supporters:

What Hollywood did better than any ad in history is promote cigarette smoking to an entire generation - and they are still doing it - resulting in how many deaths, do you think? I’d say millions. How is it we can sue asbestos companies but we don’t sue the fabulously wealthy Hollywood empire for killing so many people with their non-stop cigarette commercials which have been running in theaters and “Academy Award” winning movies for 70 years. James Dean with that cigarette hanging out of his mouth is probably responsible for more people smoking and dying prematurely than any single commercial ever put on TV or in a magazine by Malboro, Winston, Camel or Salem. He makes Joe Camel look like a rookie.

I say we bring this little hidden fact up when these overpaid pre-madonnas, who are on the screen primarily because they were born attractive, step on the stage for their Oscar with their million dollar houses and bank accounts.

How did the White people do?  I think this is a fair question since people want to bring race into this!

"you do worse than--- it---- you’re told the test does not (in this example) detect gender differences."

Come on CNN, you are a professional news organization, please no typos!

Blah, Blah, Blah. This kind of research is exemplary of how the US schools waste money on bogus pop-culture researches.

America, by far, spends the most money on education, by far, gives the most educational opportunities to all its citizens and non-citizens, blacks or non-blacks, period!

I am an immigrant.

When I grew up, kids from poor families attend school at day time, deliver newspapers before school and sell vegetables at night. Amazingly, time and time again, smart kids from that background enter top-schools and become leaders of his/her generation. These kids have to endure prejudice and unequal opportunities, but they excel over other kids who have much better economic backgrounds.

Almost all the presidents in my country and quite a good portion of business/political leaders have such poor background. So much so, we attribute poor family background as a possibly driving force behind their successes.

Black kids don't do well at school because they are on average not smart as other racial people just as Asian kids don't do well on basketball because they are shorter on average than Blacks.

Americans pretend to be blind to certain things that are so obvious to common-sense people. It's Americans' choice to distort reality, but it is sad to see that they waste previous academic resources to justify their mindless blindness.

All I can say is so what.  These were college students, they at least had to beable to read and write at a 6th grade level to get out of high school.  Shouldn't the results have all been the same?   Seems to me that we have never had an Asian or and Indian president and those kids seem to score pretty high on tests.  the only thing I see in this article is the spreading of false hope not real hope, if people really belive that the President makes you smarter I got a bridge to sell you, hopefully it won't be one built by a contractor who got the job based solely on being a "minority".

You all sound very scared of a powerful black man.  I'll pray for you.

It's amazing these articles get written.  Oh, wait it's to sell magazines.  Anything with 'O'bama' on it is selling.  Come on people!  It's time to take responsibility for your own life and what you made or created.  If it takes an African American President to wake your a-- up then be it!  People of any color needs to take responsibility for fathering kids, getting a job and doing it to their best ability, stop crying that you didn't feel like an American, stop acting like pimps and gangsters!  Be productive people of our society and we will all be better off!!!!  Americans will always respect a person who has morals, honesty, and productive!