Full confession: after the concerns raised by scientists about brain imaging, which I’ve written about here before as well as in the paper magazine, I don’t think I’ll ever look at an fMRI study the same way again. I hope I was properly skeptical about such studies before MIT's Ed Vul and colleagues showed how many of these emperors have no clothes, but now whenever a neuroimaging study crosses my desk I wonder, does it fall into the same statistical trap that so many others have, rendering the results meaningless?

So it is with what would otherwise be a perfectly interesting little study being published tomorrow in Science. It’s about envy and schadenfreude, taking pleasure from someone else’s pain, that feeling of glee we get when someone we envy suffers a setback (cf. Bernie Madoff, Wall Street bankers . . . ). The study, described in paper called “When Your Gain Is My Pain and Your Pain Is My Gain: Neural Correlates of Envy and Schadenfreude,” was led by Hidehiko Takahashi of Japan’s National Institute of Radiological Sciences, who is one of the most prolific social neuroscience imagers around (for a sampling, see here and here. Like a 2003 study finding that the psychological pain of social rejection increases activity in the same brain regions that process physical pain, this one concludes that the social and the physical are closely related. In brief, brain regions that respond to feelings of envy and schadenfreude are also those that respond to, respectively, physical pain (envy hurts) and reward/pleasure (schadenfreude feels good).

Takahashi and colleagues ran two fMRI studies on 19 volunteers. Feelings of envy (triggered by reading about a peer's status, abilities and wealth), led to heightened activity in the dorsal anterior cingulate cortex (dACC), the same region associated with the distressing aspect of physical pain. Schadenfreude (triggered by reading about the downfall of the envied person) was linked to extra activity in the ventral striatum, which processes reward.

I can hear many of you saying “so what? why shouldn’t the brain use the same circuitry to process physical pain and social pain, and physical pleasure/reward and social pleasure/reward?” At least one eminent neuroscientist is in this camp. The fact that there is heightened activity in the dACC when people felt envy “seems utterly uninformative even if true,” she said by email. “The ACC activates for nearly everything. Presumably it would also activate for any other strong emotion, not to mention cognitive effort. So who cares that it activates when you experience envy? This tells us nothing about envy and nothing about the ACC.”

In an accompanying “Perspective” in Science, UCLA’s Matthew Lieberman and Naomi Eisenberger take the opposite view, praising the significance of the study. The finding that “emotional responses to these psychological events rely on much of the same neural circuitry that underlies the simplest physical pains and pleasures” is significant and surprising, they argue. (They themselves found, in the 2003 paper I cite above, that being socially excluded raises activity in the dACC and insula, with more activity associated with feelings of greater social pain.) “Such findings suggest that the brain may treat abstract social experiences and concrete physical experiences as more similar than is generally assumed,” they write.

They offer a provocative hypothesis for how that shared circuitry for the physical and the social might have evolved, or, as they phrase the question, “Given that physical needs intuitively seem more critical to survival than social needs, why would the brain have evolved to treat them as motivationally similar?” One possibility: since newborns are totally dependent on others, “for both caregiver and infant to feel pain upon separation ensures social connection and thus offspring survival. In a sense, for mammalian infants, social needs take precedence over physical needs because meeting the social needs is what allows the physical needs to be met as well.” And after childhood, they suggest, “Being fair, cooperative, or charitable may increase the survival of the group . . . Thus, evolutionary pressures may have created internal mechanisms that register being socially cooperative as pleasurable and being ostracized as painful in order to promote the maintenance of group bonds and ensure survival.”

That just-so story may well be correct, and so might the claim of the Takahashi paper. The point the critics of neuroimaging make is that one just cannot tell from its methodology and statistical analysis. I asked scientists who were not involved in the study to take a look at it and tell me whether they thought it fell into the same methodological traps that had tripped up other fMRI studies. One key step in the analysis of the raw fMRI data, said one neuroscientist, “is unambiguously and definitively bullshit. Many of the claims in the paper are statistically suspect. Remember, this does not mean that the conclusion of the paper is wrong, simply that the data don’t provide evidence for that conclusion.”

Science received the Takahashi paper last September 8, and accepted it (after the usual peer review) on December 10. That was about three weeks before the Vul et al. criticism was widely known. I wondered whether the editors took the new concerns into account when deciding whether or not to run the paper, perhaps asking for another round of peer review in light of it. Unfortunately, none of the editors who were involved in the review and acceptance process were willing to talk to me. Instead, the journal’s spokesperson sent me this email:

“On behalf of Science, the Science press office is unable to comment on the Editorial peer review process, which is confidential. However, in general we make sure that our peer reviewers are up to date on relevant current issues regarding the papers they evaluate for us. In this case, as for all Science papers, the peer review process was rigorous and the journal took seriously any questions raised about the validity of the conclusions in the paper.”

It’s too bad the editors are unwilling to address the criticism directly, because Vul et al. call out Science and Nature as among the worst actors in the publication of problematic neuroimaging papers. If those journals believe the criticism is bunk, they should say so. But to pretend it doesn’t exist, or that eminent neuroscientists and statisticians with no axe to grind are taking the criticisms seriously, is ignoring the elephant in the room.

Update: I was remiss in not factoring in the 14-hour time difference when I emailed Dr. Takahashi asking him to respond to the criticisms of his paper. But, in the better-late-than-never spirit, let me include his response here:

"We believe Vul’s paper is misleading and their assumption was wrong. These will be rebutted in peer-reviewed journal from many researchers. . . . Our Science paper is not the case with the points they criticized (false correlations, non-independent tests). I believe that Science editors might clearly understand the potential pitfalls Vul et al pointed out. In other words, the editors might understand our paper [does not make these errors]."