Justin’s Notes

The Confession That Explains the Crisis

2026-07-23 · AI-generated insight

McIntyre catalogs the pathologies of social science from the outside — ideological infection, cherry-picking, the failure to share data, errors that "sneak through" in the direction of the researcher's hypothesis. His is a diagnostician's list, clinical and structural. What his account cannot supply is the interior experience of the person doing the p-hacking. Inzlicht supplies exactly that missing testimony.

Notice how precisely Inzlicht fills McIntyre's category. Trivers, in McIntyre's telling, found that undisclosed errors ran "most commonly in the direction of the researcher's hypothesis." McIntyre presents this as a mechanical fact about degrees of freedom. Inzlicht tells us why the errors run that way and feel harmless while they do: "Maybe I felt less wrong about the p-hacking because I was promoting social justice." The bias is not a flaw in method that discipline could scrub out — it is morally lubricated. Feeling on the right side of history was, in his words, "moral cover."

This reframes McIntyre's optimism. He hopes social science can break with its "barbarous past" the way medicine did, through better method — experimentation, replication, data sharing. But medicine's reforms fought against ignorance and inertia, not against the researcher's own sense of virtue. Inzlicht reveals an adversary that better protocols alone cannot reach: a scientist who does not experience himself as cheating because the cheating serves the good.

The Coon anecdote and Reich's essay show the same mechanism from the receiving end. The anthropologists condemned Putnam's book without reading it, confident they were defending the vulnerable. Reich watches the same reflex — the fear that inquiry sits on a slippery slope toward atrocity — foreclose questions before evidence is examined. In each case the moral stakes don't merely coexist with bad epistemics; they authorize them. McIntyre asks how to fix the social sciences. Inzlicht's confession suggests the hardest fix is not statistical but the recognition that righteousness is precisely when one is least inclined to check one's work.

Woven from these notes

The orthodoxy maintains that the average genetic differences among people grouped according to today’s racial terms are so trivial when it comes to any meaningful biological traits that those differences can be ignored. The orthodoxy goes further, holding that we should be anxious about any research into genetic differences among populations. The concern is that such research, no matter how well-intentioned, is located on a slippery slope that leads to the kinds of pseudoscientific arguments about biological difference that were used in the past to try to justify the slave trade, the eugenics movement and the Nazis’ murder of six million Jews. I have deep sympathy for the concern that genetic discoveries could be misused to justify racism. But as a geneticist I also know that it is simply no longer possible to ignore average genetic differences among “races.”
— David Reich How Genetics Is Changing Our Understanding of 'Race'
When so many studies fail to be replicated, or draw different conclusions from the same set of facts, it does not instill confidence in the social sciences. Whether this is because of sloppy methodology, ideological infection, or other problems, the result is that even if there are right and wrong answers to many of our questions about human action, most social scientists are not yet in a position to find them. It is not that none of the work in social science is rigorous enough, but when policymakers (and sometimes even other researchers) are not sure which results are reliable, it drives down the status of the entire field. If medicine could break with its barbarous past, isn’t the same path open to the social sciences? For years, many have argued that if they could emulate the “scientific method” of the natural sciences, they too could become more scientific. But this simple advice faces several problems. Among the issues that plague contemporary social scientific research: Too much theory: A number of social scientific studies propose answers that have not been tested against evidence. The classic example here is neoclassical economics, where a number of simplifying assumptions — perfect rationality, perfect information — resulted in beautiful quantitative models that had little to do with actual human behavior. Lack of experimentation/data: Except for social psychology and the newly emerging field of behavioral economics, much of social science still does not rely on experimentation, even where it is possible. For example, it is sometimes offered as justification for putting sex offenders on a public database that doing so reduces the recidivism rate. This must be measured, though, against what the recidivism rate would have been absent the Sex Offender Registry Board (SORB), which is difficult to measure and has produced varying answers. This exacerbates the difficulty in (1), whereby favored theoretical explanations are accepted even when they have not been tested against any experimental evidence. Fuzzy concepts: Some social scientific studies can lead to misleading conclusions because of the use of “proxy” concepts for what one really wishes to measure. A recent example includes measuring “warmth” as a proxy for “trustworthiness,” in which researchers assumed — on the basis of studies which show that we are more likely to trust someone whom we perceive to be “on our side” — that perceptions of scientists as “cold” meant that they would be less trustworthy as well. But the two concepts may not be interchangeable. Ideological infection: This problem is rampant throughout the social sciences, especially on topics that are politically charged. Two ongoing examples are the bastardization of empirical work on the deterrence effect of capital punishment and the effectiveness of gun control on mitigating crime. If one knows in advance what one wants to find, one will likely find it. Cherry picking: The use of statistics allows multiple “degrees of freedom” to scientific researchers, but this is the most likely to be abused. In studies on immigration, for instance, a great deal of the difference between them is a result of alternative ways of counting the “costs” incurred by immigration. This is obviously also related to (4) above. If we know our conclusion, we may shop for the data to support it. Lack of data sharing: As the evolutionary biologist Robert Trivers reports in Psychology Today, there are numerous documented cases of researchers failing to share their data in psychological studies, despite a requirement from APA-sponsored journals to do so. When data were later analyzed, errors were found most commonly in the direction of the researcher’s hypothesis. Lack of replication: Psychology is undergoing a reproducibility crisis. One might validly argue that the initial finding that nearly two-thirds of psychology studies were irreproducible was overblown, but it is nonetheless shocking that most studies are not even attempted to be replicated. This can lead to difficulties, where errors can sneak through. Questionable causation: It is gospel in statistical research that “correlation does not equal causation,” yet some social scientific studies continue to highlight provocative results of questionable value. One recent sociological study, for instance, found that matriculating at a selective college was correlated with parental visitation at art museums, without explicitly suggesting that this was likely an artifact of parental income.
— Lee McIntyre To Fix the Social Sciences, Look to the “Dark Ages” of Medicine
Carleton Coon was one of the greatest anthropologists of the 20th century and a champion of the study of racial differences. In 1961 he was elected president of the American Anthropological Association. It was during this year that Carleton Putnam's infamous book Race and Reason came out. Putnam's book argued that there were significant differences between the races and that this had important political implications. A few anthropologists became upset when they saw the book gain popularity and wanted the AAA to intervene. But they knew that Coon supported the book and that, as a result, having the AAA take action against it would be difficult. So they organized a secret meeting behind Coon's back in order to issue a statement denouncing Putnam's work. Coon found out about the meeting and went to stop it. Upon entering the meeting Coon asked all the association members who had actually read Putnam's book to raise their hands. Only one rose. He then asked for the hands of everyone that had even heard of the book prior to that meeting. Only a few hands rose. None the less, the statement against the book passed. Coon, disgusted by the actions of his peers, resigned from the AAA (1). This incident stands out in the history of science as a particularly clear example of scientists not living up to what people expect of them. People often take what scientists say for granted. They trust that scientists have looked at the evidence rationally and are giving the public as accurate an account of all the relevant facts as they can. When explaining science to laypeople researchers are expected to only make authoritative statements on topics they are knowledgeable of, to not lie about scientific evidence, and to not omit obviously relevant facts. People trust scientists to do these things and so feel comfortable taking what they tell them for granted. (See
My work was uplifting people, or so I told myself. Looking back, I suspect that feeling like I was on the right side of history was a kind of moral cover. Maybe I felt less wrong about the p-hacking because I was promoting social justice. The conclusions I drew were what the good guys already believed, after all.
— Michael Inzlicht My P-Hacking Felt Like Truth