Antivaxxer Steve Kirsch unknowingly identifies the fatal flaw of an antivax “report” from the McCullough Foundation

Before I discuss how Kirsch inadvertently nailed the key problem with the McCullough Foundation report, I must point out that I do feel some empathy for him, as last year he suffered a major health issue. Last year, he suffered a subretinal hemorrhage that resulted in major vision loss in one eye, caused, according to him, “either a central serous retinopathy (CSR) that burst (which started when I was under a lot of stress with all the work I was doing), or dry age-related macular degeneration (AMD) that progressed to wet AMD.” Unsurprisingly, he’s been attracted to quackery, such as claims that DMSO drops might restore his vision, although in fairness he hasn’t tried it, as far as I can tell, and remains somewhat skeptical. More interestingly to me, he has actually wondered in an August 2024 video whether taking The Wellness Company’s “Spike Detox” formula might have contributed to his subretinal hemorrhage, even referring to it as an “expensive lesson” to check the contraindications, the “expensive lesson” being the loss of vision in one eye:

The lesson that I learned is if you take any supplements, be sure you know why you’re taking the supplements, and don’t take supplements you don’t need. So always look for counterindications for anything that you’re taking… [41:10] This thing happened to me. And I’ll people know to be careful, to be careful about anything you put into your body. I don’t care if it’s a vaccine or a supplement or a drug. Make sure you really understand the interactions. And make sure that all of your specialists know what you’re taking, so that you can look for things that are probably not going to provide you any benefit whatsoever, and would only contribute downside risk. Because I think that the—I was on the Peter McCullough protocol, which I think is a great protocol, ! mean it’s highly highly effective for preventing blood clots, and for eliminating any blood clots that you might have. It turned out that wasn’t my problem. Because I was just taking it thinking that, you know there was no harm in taking it. But because I had a retinal condition and because for whatever reason my doctor didn’t realize that I was taking these three supplements, my retinal bleed was the mother of all retinal bleeds. And the first thing that happened when I ran into my – the first surgeon that I saw, he said what blood thinners are you on? And I said I’m not on any blood thinners but I think I know what you’re talking about.

Oddly enough, this isn’t bad advice, for the most part. I’ve written about the Spike Detox Formula (which consists mostly of an enzyme called nattokinase) before and how it is quackery being peddled by The Wellness Company, for which Dr. McCullough serves as Chief Scientific Officer and spokesdoctor. While I highly doubt that this supplement was the cause of Kirsch’s subretinal hemorrhage, I do not doubt that it is antivax COVID-19 quackery, which makes it odd to me that Kirsch is so enthusiastic about the McCullough Foundation report.

That digression aside (seriously, I’m starting to think that I have more and more in common with Molly Conger and her digressions on her podcast), let’s take a look at what Kirsch says about the McCullough Foundation Report in a Substack entry called New McCullough autism paper: Vaccines are the single largest contributor to the risk of autism. Naturally, his tagline for this Substack is, “No surprises. Of course, this paper will be ignored, censored, and ‘fact checked.’” Again, the ironic thing is that Kirsch hits the nail on the head regarding the fatal flaw of this report without even realizing that he’s done so. Let’s begin.

After summarizing the report and saying, “In short, it’s the vaccines,” Kirsch decides that he has to tell his readers what is superior about this report compared to the decades’ worth of studies, meta-analyses, and systematic reviews that conclude that vaccines do not increase the risk of being diagnosed with autism, calling it a “meta-collection” of “roughly 100 – 150 citations on non‑vaccine factors (e.g., prenatal infection, maternal age, pollution) and another ≈ 130 papers on vaccination or excipients” that were “then merged…into thematic tables.” (Hint: This is not the way you do systematic reviews.)

Amusingly, this leads Kirsch to correctly identify that different methods can lead to different conclusions:

The divergence arises from both methodology and interpretive framework.

a. Different Inclusion Rules

Yep! Kirsch nailed it! If you use a different evidence hierarchy that includes, along with randomized, double-blind studies, and large registry studies, low quality evidence such as non-replicated mechanistic papers, uncontrolled case-series and ecological correlations, and in vitro cell culture work that has never been shown to apply to in vivo models, you’re going to come to different conclusions! Here’s the thing. For any common disease or condition, there will always be hundreds, if not thousands, of lab-based studies in cell culture that propose mechanisms of etiology, of how the condition is caused. Inevitably, many of them will be red herrings or dead ends and won’t lead to an actionable understanding of the biology of how the condition or disease state is caused or how it can be treated. By design, the McCullough Foundation Report dilutes the high quality preclinical, clinical, and epidemiological science with much lower quality preliminary reports, uncontrolled observations, and the like, all in order to be able to conclude that vaccines cause autism.

Kirsch is so close to understanding this, but unfortunately he seems to see this feature (and not a bug) of the McCullough method as a good thing, as you can see in his next bullet point:

b. Interpretive Approach

– Lancet/NIH treat correlation ≠ causation; they require population‑level replication before causal language.

– McCullough interprets overlapping mechanistic signals (immune activation, metal toxicity, mitochondrial deficit) as causal chains even without direct cohort confirmation.

Once again, what Kirsch is basically saying is that, contrary to what the Lancet/NIH method demanded, McCullough accepts lower quality, unreplicated, and uncontrolled evidence as causative without any actual rigorous evidence of causation. Moreover, if you peruse the list of 308 references, you’ll find just how low quality some of the evidence cited is. For example, reference #7 is Wakefield’s retracted 1998 Lancet case series! Also included are articles and studies from a panoply of antivax “scientists” about whom I’ve written much over the years, in some cases going back two decades both here and at my not-so-super-secret other blog, including David Kirby, Andrew Zimmerman, Mark and David Geier, Brian Hooker, Christopher Shaw and Lucija Tomljenovic, James Lyons-Weiler, Gary Goldman, Christopher Exley, Catherine DeSoto and Robert Hitlan, Sallie Bernard and Lynn Redwood, Stephanie Seneff, Gayle DeLong, Mary Holland (an antivax lawyer), Theresa Deisher, Helen Ratajczak, Mark Blaxill and Boyd Haley, and Anthony Mawson. They even included a citation (#281) of an article by Robert O. Young, one of the quackiest quacks about whom I’ve ever written, a mail-order trained naturopath who doesn’t believe in germ theory, denies that sepsis is caused by bacterial infection, thinks that viruses are “molecular acids,” and claims that cancer is in reality too much acid in the body, that cancer is an acidic liquid, and that that tumors are the body’s natural defense of walling off “spoiled” tissue. They even cite multiple articles from Peter McCullough and his antivax authors citing themselves (e.g., John Leake and Nicholas Hulscher) and the rabidly antivax org Children’s Health Defense, including an infographic entitled Vaccines Can Cause Autism! (Indeed, Figure 2 comes from the infographic.)

That’s what I would call a failure of selection criteria for studies to be cited and discussed in the review. Of course, it’s intentional. These quack studies are cited along with legitimate studies as though the quack studies are just as valid and cast doubt on the findings of rigorous scientific studies. That’s the point.

Again, Kirsch comes so close to getting it…without getting it:

c. Selection Bias & Data Weighting

The McCullough group weights:

• in‑vitro data showing neuronal toxicity of ethyl‑Hg or Al;
• ecological trends (autism incidence vs. vaccine dose counts);
• clinical case series of “regressive autism after vaccination.”

Mainstream commissions discount those sources as uncontrolled and biologically non‑comparable to real pediatric exposure levels.

Notice the wording, “Mainstream commissions dismiss.” Kirsch doesn’t get it while almost getting it. Mainstream scientists do not count such data as compelling because they are, in fact, from uncontrolled studies and often using doses of compounds that aren’t comparable to real pediatric exposure levels. As for studies showing neuronal toxicity of ethyl mercury or aluminum, the concentrations used are basically always much higher than what could be expected or achieved even in the bloodstream of an infant, basically because the amount in vaccines is so tiny (or was so tiny in the case of ethyl mercury, given that thimerosal, the preservative containing it, was mandated to be removed from childhood vaccines in 1999 and was removed by early 2002).

Moreover, if there’s one rule about biological plausibility, it’s that any mechanism or toxicity observed in cell culture that doesn’t translate to humans in well-designed epidemiological studies or randomized controlled trials is incredibly unlikely to be important. For example, there is a passage in the McCullough Report (pp. 29-30 in which the authors discuss the different pharmacokinetics (the rates at which a drug level increases and then the drug is eliminated from the body) of ethyl and methyl mercury to try to imply that ethyl mercury is actually more toxic than methyl mercury (which is known to be toxic). Even if this were all true, as a practical matter it isn’t relevant to the question being asked. Large, well-controlled epidemiological studies consistently show that exposure to ethyl mercury in thimerosal in vaccines at the cumulative dose that infants received even when the most thimerosal-containing vaccines were included in the vaccine schedule, is not associated with a higher risk of autism. Mercury in vaccines at the doses used then did not cause autism, and, given that mercury was removed from almost all childhood vaccines nearly a quarter century ago, mercury in vaccines does not cause autism now. It’s now a moot point.

Then, of course, once again Kirsch is impressed by ecological data and uncontrolled case series, seemingly more than well-designed large epidemiological studies that appropriately control for confounders. In this, he sounds very much like Brian Hooker, who while giving a talk on his “reanalysis” of a large study that failed to find a correlation between MMR vaccination and autism, said:

So I reanalyzed the dataset using what is a very, very simple statistical technique. I think that in statistics simplicity is elegance. And I’m not really that smart; so I like to do simple, easy things rather than much more intellectually challenging things. So I did the simplest, most straightforward analysis, which is a Chi Squared analysis…

Here’s a hint: In statistics and epidemiology, the simplest analysis is often not the correct analysis, and, boy, was this the case for Hooker’s reanalysis of the DeStefano et al dataset! He didn’t control for simple confounders. He did a crappy statistical analysis. He botched the analysis in pretty much every way possible. And Kirsch—as well as, apparently, McCullough, Wakefield, and the rest of his antivax co-authors—is more impressed with “simplicity.” Of course, it is a common antivax conspiracy theory that appropriate correction for confounders that causes a correlation between vaccines and autism to disappear in dataset is a conspiracy in which “they” are covering up a link.

Indeed, in my first post that ever went viral, I deconstructed all the misinformation and conspiracy mongering shortly after, as did others, who noted that the entire Simpsonwood transcript was not at all consistent with any effort to “cover up” a link between mercury and autism, but rather included a debate about how best to deal with confounders in a study by Verstraeten et al. (Of course, to antivaxxers, any adjustment for confounders that eliminates a link between vaccines and whatever bad health outcome that they attribute to vaccines is always a “coverup,” not science doing what science is supposed to do and being rigorous.) Years later, I referred to this as the Simpsonwood conspiracy theory and, sometimes, part of the central conspiracy theory of the antivaccine movement, in which They (the CDC, government, FDA, medical profession, big pharma, and who knows who else—aliens, possibly) “know” that vaccines cause autism but actively work to cover up any incriminating evidence.

Finally, Kirsch notes:

They agree on

Both camps agree on advanced parental age, premature birth, maternal infection, and high inflammation markers as consistent ASD risk factors.

Both accept a role for mitochondrial and redox biology in some subgroups.

Yes, and no. It’s the way that antivaxxers “accept a role for mitochondria and redox biology in some subgroups.” Does anyone remember all the redox pseudoscience and woo that came out of the Hannah Poling case, where antivaxxers claimed that mitochondrial disorders plus vaccines equals autism from c.2008-2012?

As for the rest:

They differ solely on iatrogenic exposures

Mainstream bodies see no epidemiologic signal once bias is controlled.

The McCullough report emphasizes uncontrolled subpopulation findings.

The difference arises from how each group defines valid evidence and how strictly they apply epidemiologic causal criteria.

I actually can’t argue with that. Antivaxxers like McCullough have a predetermined outcome that they seek to find evidence for that lets them support that outcome—or at least lets them seem to support that outcome to people without a background in clinical or epidemiological research. They are therefore quite willing not to define “valid evidence” or apply “epidemiological causal criteria” very strictly at all. “Mainstream bodies” don’t see any epidemiological signal once bias is controlled for, and that is the very reason why antivaxxers like McCullough, Wakefield, and Kirsch value evidence that is uncontrolled, or at least much less well-controlled than standard epidemiological rigor normally demands.

Whom do you believe?

When you come right down to it, it all boils down to a single question, at least according to Steve Kirsch: Whom do you believe? Here’s how he puts it:

The million dollar question: Which is more likely to be correct?

It depends on which evidence bases you value more.

When you go to an AutismOne Conference, and the speaker asks the parents, “How many of you believe your child’s autism was linked to vaccines?” and nearly every hand in the room goes up, that’s a little hard to explain.

No, it’s not. Not really particularly given that Autism One is an antivax conference. Think about it. The vast majority of parents who are willing to pay to travel to the suburbs of Chicago to attend such a conference will, at the very least, lean in the direction of believing that vaccines caused their child’s autism. Having studied and written about the conference dating back nearly two decades, I know that the vast majority of attendees are fervent antivax true believers looking for ammunition to support their belief and treatments based on the dubious science used by the presenters at the conference.

Indeed, those of us who’ve been paying attention long enough know that Autism One actually tries very hard to keep skeptics and reporters not aligned with the antivax cause, particularly skeptics and reporters who might write news stories and critical blog posts about the talks and “science” presented there. Indeed, in 2011, Jamie Bernstein, Vice President of the Women Thinking Free Foundation and a vaccine advocate, registered for Autism One along with Ken Reibel, in order to see what the antivaccine movement was saying there. That’s not all, as Liz Ditz wrote in 2011, a number of reporters and those not aligned with the antivax movement who tried to attend Autism One and report on its proceedings have been ejected. Indeed, the organizers called the police on Bernstein and Reibel, who were escorted out by security guards. I’d be willing to bet that Steve Kirsch doesn’t know this history, but I do, having followed it for decades.

No, it’s not difficult at all to explain why damned near every single parent of an autistic child (or of autistic children) who attended Autism One believes that vaccines caused their children’s autism in some way.

In the end, Kirsch is almost correct without actually being correct. He notes that the McCullough Report mixes low quality preliminary preclinical studies, epidemiological observations not corrected for confounding, and uncontrolled case series with “mainstream” evidence that values controlling for confounders, confirming mechanistic observations from preclinical cell culture and animal studies with clinical and epidemiological studies in humans, and, in general, scientific, epidemiological, and clinical rigor over anecdotal experience. Don’t get me wrong. Uncontrolled case series and mechanistic studies in preclinical models can be good for generating hypotheses, but those hypotheses must be tested confirmed in larger, more rigorous human studies. Moreover, many of the preclinical studies cited have been tested in more rigorous human studies and been found to be wanting. McCullough and his coauthors likely know, but refuse to accept, that simple fact. They can thus weave all sorts of mechanistic speculations—which I like to refer to as “biobabble,” as a nod to the famous term about some Star Trek technology, “technobabble”—with low quality clinical and epidemiological data to paint a picture of plausibility that seems compelling if you don’t have a strong background in science.

One final thought I had. What Kirsch is proposing might, on its surface, seem to be science-based medicine (SBM). After all, we have long argued that basic and preclinical science findings should be included in any evaluation of a medical treatment or causation claim, particularly for alternative medicine. Here’s the difference. What SBM proposes is to use basic science findings to increase scientific rigor. In other words, we like to point out that homeopathy is physically impossible, that, for homeopathy to be true, huge swaths of well-established scientific findings in physics, chemistry, and biology would have to be not just wrong, but spectacularly wrong. In other words, we propose that what we know about science will primarily cast doubt on fantastical claims. In contrast, what Kirsch seems to be doing is to argue that all sorts of preliminary and contradictory basic science and preclinical data should be brought in to cast doubt on well-founded clinical and epidemiological findings by claiming that, even though we don’t find a hint of a whiff of a suggestion in datasets and large, well-designed clinical trials and epidemiological studies that vaccines increase the risk of autism, some study in which mercury or aluminum is put into the culture medium of neurons in a dish at a high concentration causes toxicity should cast doubt on all of that. I hope you can see the difference.

In the end, your answer to Kirsch’s final question—which evidence bases you value more—determines whether you have a scientific mindset or not. If you value high quality evidence and understand the hierarchies of rigor in scientific evidence, you have a scientific mindset. If you do not, you are more like Steve Kirsch or Peter McCullough—or are at least likely to be susceptible to the kinds of misinformation they spread.