Parents and policy analysts are suffering from whiplash after watching the Food and Drug Administration get excited about, then ultimately abort, authorization of Pfizer’s Covid-19 vaccine for children aged six months to four years. The series of events is unprecedented and unnecessary, and it further undermines the agency’s credibility at precisely the moment the public needs faith in vaccine regulators.
The FDA initially gave Pfizer a layup. The company did not have to run a large trial showing the vaccine lowers the risk of severe outcomes; instead, the agency would settle for a trial that showed that antibody levels generated in kids were close enough to levels generated in older age groups—a type of study design called “non-inferiority,” which has been faulted for being overly permissive.
Pfizer succeeded in meeting this mark for kids aged five to 11 with a ten-microgram dose, leading to emergency-use authorization (EUA) at those ages. It tried again in the six-month to four-year-old cohort with a lower dose—three micrograms (the adult dose is 30). Unfortunately, in December 2021, Pfizer announced that its trial fell short, and modified it to see whether a third jab could get it to the mark.
In late January, something unusual happened. While investigators were still testing the third dose, Scott Gottlieb, a current Pfizer board member and former FDA commissioner, suggested on CBS Face the Nation that federal officials were considering approving the two doses while waiting for the ongoing trial. In fact, just three days later, the agency asked Pfizer to submit whatever results it had for consideration of granting EUA.
Inviting a company to submit the results of an ongoing trial that had already failed to meet its mark was a bizarre, even unprecedented, decision. Paul Offit, director of vaccine communication for the Children’s Hospital of Pennsylvania, said, “It doesn’t make sense we would approve a two-dose vaccine on the assumption the third dose would make up for deficiencies of the two doses.”
In the days that followed, more information emerged from anonymous leaks. Another measure (or endpoint) of the vaccine’s efficacy, the number of cases of SARS-CoV-2, varied between the recipients; fewer kids who got the vaccine tested positive for the virus. News reports said that the reduction in cases was 57 percent. A frenzied discussion online ensued. Observers speculated that the number of events must be low, so the 57 percent reduction should be taken with a grain of salt. The true reduction could be much higher or lower—or, in statistical parlance, the 95 percent confidence interval was likely wide. Others worried that it would be important to see the breakdown of how many infections were due to the Delta and Omicron variants. After all, Omicron was the dominant variant in the moment, and three studies had shown that the vaccine had poor effectiveness against Omicron.
In any case, ambiguities remained even with this secondary endpoint. An important question was whether the kids who tested positive were symptomatic. During the holiday season, many families traveled or tested asymptomatic children to try to foster safer gatherings. Could some of the difference in cases be due to that massive testing? This mattered, because the goal of vaccination is to reduce the harms of the virus—feeling bad, being hospitalized, or worse—not merely to attain fewer asymptomatic positives.
A vaccine advisory meeting was scheduled for February 15, and many waited eagerly for dissemination of trial data, which were slated to be made public the Friday beforehand (February 11). On February 10, however, more leaked data emerged. The New York Times reported the sample size of the study, and that there were just 50 total infections, with a 57 percent reduction in cases. These data were sufficient for statisticians to estimate roughly the confidence interval, which was going to be very wide (anywhere from 25 percent to 75 percent), prompting more debate about whether it was prudent for the FDA to approve the shot.
Then, in a stunning about-face, Pfizer announced on February 11 that it was pulling its EUA from consideration. Gottlieb told reporters that there were too few cases, and too few symptomatic cases, in the trial. The numbers were unreliable, and the FDA and Pfizer would wait for the third-dose results to come in. Some observers were upset and argued that the vaccine should be made available, or that at least all data be made public—the latter an entirely reasonable request.
Few have grappled with the implications for this series of events, which represents an unforced error in vaccine regulation. First, the FDA vaccine division is operating at diminished capacity. Marion Gruber and Philip Krause, the director and deputy director of that division, famously resigned in the fall of 2021 after decade-long careers in response to pressure from the White House to approve boosters for all adults. As such, trust is already low in some quarters. Second, vaccine regulation is a reputational field. To an outside observer, it looks as if the agency does not know what it is doing. The agency has asked Pfizer to show that antibody levels are non-inferior—does it want this information or not? If the agency is happy with a reduction in Covid cases, it could have explicitly asked Pfizer to make this the primary endpoint of the study.
But it is unscientific to run a trial and then accept any endpoint that looks favorable. Scientists call it the Texas Sharpshooter fallacy: someone fires indiscriminately at the side of a barn and then paints a bullseye where several bullets happened to bunch together. The FDA appeared to be inviting such gaming.
Worse, just two weeks later, important data emerged suggesting that the vaccine was much less effective at blocking infection in the older, five- to 11-year-old age group. Investigators speculate that one potential reason was the lower dose. But if the dose is responsible for the inferior performance in five- to 11-year-olds, what would have happened had authorities authorized an even lower dose in those aged six months to four years?
The American public witnessed a two-week media cycle in which some parents became excited by the prospect of imminent approval, while others grew concerned at the possibility of school or daycare mandates based on a study that had failed to meet its primary endpoint. As an expert in drug regulatory policy and author of two books on this topic, I have never witnessed anything comparable to this episode. At least some of the dysfunction may be due to a lack of leadership in vaccine products within the agency. Some experts praised the FDA for making the right call in the end and deciding not to make the shot available. But this is like cheering an arsonist for putting out a fire he himself started. The federal government is spending the FDA’s credibility like monopoly money at precisely the time we need faith in vaccine regulators.
Photo by Matt Hunt/SOPA Images/LightRocket via Getty Images