The Differences Between the Vaccines Matter
By Hilda Bastian, The Atlantic
08 March 21
Yes, all of the COVID-19 vaccines are very good. No, they're not all the
same.
Public-health officials are enthusiastic about the new, single-shot COVID-19
vaccine from Johnson & Johnson, despite its having a somewhat lower efficacy
at preventing symptomatic illness than other available options. Although
clinical-trial data peg that rate at 72 percent in the United States,
compared with 94 and 95 percent for the Moderna and Pfizer-BioNTech
vaccines, many experts say we shouldn't fixate on those numbers. Much more
germane, they say, is the fact that the Johnson & Johnson shot, like the
other two, is essentially perfect when it comes to preventing the gravest
outcomes. "I'm super-pumped about this," Virginia's vaccine coordinator told
The New York Times last weekend. "A hundred percent efficacy against deaths
and hospitalizations? That's all I need to hear."
The same glowing message-that the COVID-19 vaccines are all equivalent, at
least where it really counts-has been getting public-health officials and
pundits super-pumped for weeks now. Its potential value for promoting
vaccination couldn't be more clear: We'll all be better off, and this
nightmare will be over sooner, if people know that the best vaccine of all
is whichever one they can get the soonest. With that in mind, Vox has urged
its readers to attend to "the most important vaccine statistic"-the fact
that "there have been zero cases of hospitalization or death in clinical
trials for all of these vaccines." The physician and CNN medical analyst
Leana Wen also made a point of noting that "all of the vaccines are
essentially a hundred percent" in this regard. And half a dozen former
members of President Joe Biden's COVID-19 Advisory Board wrote in USA Today,
"Varying 'effectiveness' rates miss the most important point: The vaccines
were all 100% effective in the vaccine trials in stopping hospitalizations
and death."
There's a problem here. It's certainly true that all three of the
FDA-authorized vaccines are very good-amazing, even-at protecting people's
health. No one should refrain from seeking vaccination on the theory that
any might be second-rate. But it's also true that the COVID-19 vaccines
aren't all the same: Some are more effective than others at preventing
illness, for example; some cause fewer adverse reactions; some are more
convenient; some were made using more familiar methods and technologies. As
for the claim that the vaccines have proved perfectly and equally effective
at preventing hospitalization and death? It's just not right.
These differences among the options could matter quite a bit, in different
ways to different people, and they should not be minimized or covered over.
Especially not now: Vaccine supplies in the U.S. will soon surpass demand,
even as more contagious viral variants spread throughout the country. In the
meantime, governors are revoking their rules on face masks, or taking other
steps to loosen their restrictions. It's tempting to believe that a simple,
decisive message-even one that verges on hype-is what's most needed at this
crucial moment. But if the message could be wrong, that has consequences.
The idea that all of the vaccines are pretty much the same, in that they're
perfect at preventing COVID-19 hospitalizations and death, got its legs on
social media. The USA Today op-ed by the former members of the Biden team
illustrated this by linking to a data table found on Twitter. Created by the
infectious-disease doctor Monica Gandhi, it showed a variety of trial
results for six different vaccines. One column was rendered in canary
yellow-"Protection from hospitalizations/death"-and every cell read "100%."
A similar table, tweeted out a few days earlier by the dean of Brown
University's School of Public Health, Ashish Jha, conveyed the same idea
through a grid of zeros-as in, zero people hospitalized, zero people dead.
The prominent physician and researcher Eric Topol followed with his own
clinical-trial data summary featuring a column of 100 percents. "That is
impressive!" he wrote across the top. All told, their posts would be
retweeted about 15,000 times.
The data were indeed suggestive of an encouraging idea. Based on the numbers
so far, we can expect the vaccines to provide extremely high levels of
protection against the most dire outcomes. Still, we don't know how high-and
it's clear they won't uniformly cause hospitalizations and deaths from
COVID-19 to disappear in vaccinated people.
The experts understand this, of course. Gandhi has been updating her table
as more data come in, and now pegs Moderna's efficacy on that front at 97
percent; Jha has since tweeted that "nothing is 100 percent . But these
vaccines sure are close"; and Topol told The Atlantic that the numbers in
his tweet are not a sufficient basis from which to draw "any determination
of magnitude of effect," though the fact that they all point in the same
direction is "very encouraging." Still, the message of perfection that their
initial tables and tweets spawned-the gist, for many readers, of all those
100s and zeros-has since been picked up far and wide, and misinterpreted
along the way.
To grasp the shaky nature of these particular data, it's important to
remember how the vaccine-development process began. Last April, not long
after the pandemic began, the World Health Organization set out a target
efficacy for vaccines of 50 percent, with options for how that value should
be measured. A vaccine could be shown to reduce the risk of symptomatic
disease, severe disease, or transmission of the coronavirus. The FDA offered
similar guidance in June, and other regulatory agencies also followed the
WHO's lead. Among these choices, symptomatic disease was the most feasible,
because it's both a common outcome and one that's easier to confirm in a
large-scale trial. An outcome that included asymptomatic infections would
have been even more common, but screening for all infections would have been
prohibitive, especially early in the pandemic. So that's how the vaccine
trials were designed: Each would try to demonstrate at least 50 percent
efficacy with respect to symptomatic disease as its "primary outcome."
The trials could have used severe disease, hospitalization, or death as
primary outcomes, but that would have slowed things down. These events are
far more infrequent-there could have been 200 infections for each COVID-19
death in the U.S.-and that means it would have taken more time, and larger
numbers of trial participants, to generate enough data to be sure of a 50
percent efficacy. Developers did include "severe COVID-19" as a secondary
outcome-that is, one that would be measured and analyzed, but for which the
trial might not have been designed to provide a definitive answer. Efficacy
against hospitalization and against death, however, were not included as
secondary outcomes for every trial.
Given that fact, the data can't support a claim that the vaccines are 100
percent effective at preventing these serious outcomes. (Topol highlighted
this very issue in an op-ed last fall for The New York Times.) Out of the
six vaccines included in the dramatic data tables that made the rounds on
Twitter, the clinical trials for only two of them-Oxford-AstraZeneca's and
Johnson & Johnson's-included hospitalization for COVID-19 as a secondary
outcome, and reported that efficacy rate. The clinical research for one
other vaccine, made by Novavax, had hospitalization as a secondary outcome,
but that trial hasn't been reported in full yet. (On my website, I've
provided more detailed information and analysis of the relevant data.)
Now, a casual reader of clinical-trial reports-or their summaries on social
media-might take the fact that no hospitalizations of vaccinated people are
mentioned to mean that none occurred. That's risky, given that pieces of the
data have been published across various medical journals and via several
different regulatory agencies rather than in full in one place; that the
plans for some trials did not specify ahead of time that the vaccine's
efficacy at preventing hospitalizations would be calculated; and that we've
seen only minimal early data (via a press release from Novavax) from one of
them. It would be just as risky to assume that all hospitalizations would be
included in the analyses of people who developed severe COVID-19.
Hospitalization and severe disease are not synonymous-people could be coping
at home even though COVID-19 has caused their oxygen levels to drop
severely, and moderately ill people might be hospitalized out of an
abundance of caution when they are at high risk of getting worse.
The two vaccine trials that did explicitly report hospitalizations as an
efficacy outcome make this latter issue very clear. For the AstraZeneca
vaccine, one person in the control group had severe COVID-19, but eight
people were hospitalized; for Johnson & Johnson, 34 people in the placebo
group had severe COVID-19, but only five people were hospitalized. It's true
that zero vaccinated people were hospitalized in either study after the
vaccines took effect. But with numbers that small, you can't draw a reliable
conclusion about how high efficacy may be for these outcomes. As Diana
Zuckerman of the National Center for Health Research pointed out about the
Johnson & Johnson trial, "It's misleading to tell the public that nobody who
was vaccinated was hospitalized unless you also tell them that only 5 people
in the placebo group were hospitalized." She's right. And you can't be
confident about predicting effectiveness precisely in a wider population
outside the trial, either. For example, some of the vaccine trials included
relatively few people older than 60 as participants.
You can see how fragile these numbers are by looking at those compiled for
severe disease. In the Pfizer trial, for example, just one vaccinated person
developed severe COVID-19 versus three in the placebo group-which meant that
a single bout of disease made the difference between a calculated efficacy
rate of 66 percent and one of 100 percent. For the Novavax and
Oxford-AstraZeneca trials, there were zero people with severe disease in the
vaccinated group versus only one in the control group, so adding or
subtracting one would have been even more dramatic. The problem is even
greater for deaths. For that efficacy analysis, only two of the vaccine
trials-for Moderna's and Johnson & Johnson's-reported any COVID-19 deaths at
all in the control groups.
It's also important to remember that these are early results: Some people
who enrolled very late in the trials aren't yet included in reported data,
and analysis is still under way. Indeed, the FDA pointed out in December
that one vaccinated person in the Moderna trial had been hospitalized with
apparently severe COVID-19 two months after receiving a second dose. That
person was in a group still awaiting final assessment by the researchers,
and was not mentioned in Moderna's formal readout of results.
We've learned a little more from the ongoing public vaccination programs.
Four important reports have come in the past two weeks. In one, researchers
compared about 600,000 people who had had a full course of the Pfizer
vaccine in Israel with 600,000 people matched in age and other demographics
who had not been vaccinated. The shots' effectiveness at preventing
hospitalization was measured at 87 percent. ("This vaccine is fabulous in a
real world setting," Jha tweeted in response.) A preprint from Scotland
reported an efficacy rate against hospitalization of about 80 percent among
people 80 or older, almost all of whom had received only one dose of either
the Pfizer or the AstraZeneca vaccine. Two reports from Public Health
England estimated a reduction of hospitalization of about 50 percent and 43
percent for the same age group, again almost all after just one dose of the
Pfizer vaccine. These are exciting outcomes-those vaccines really, really
worked! But they oughtn't lead anyone to think that the vaccines are all the
same, and that protection will be perfect.
Where does that leave us for making decisions? As Anthony Fauci told The New
York Times last weekend, "Now you have three highly effective vaccines.
Period." Again, you will get a lot of benefit from any of them, and your
risk will shrink even more as those around you get vaccinated too. Whichever
one you start with, a booster may be coming in the not-so-distant future, of
the same vaccine or perhaps a different one. By taking the first vaccine you
can get, you'll also avoid the risk of finding yourself without protection
if infection rates surge where you live.
Efficacy is merely one layer, though. The Pfizer and Moderna vaccines have
an edge at preventing symptomatic illness, but the Johnson & Johnson vaccine
brings its own advantages. It has no demanding freezer requirements, which
means it's easier to distribute and more accessible to many communities.
It's more affordable than the other two-the company is providing it at cost
around the world. Then there's the fact that resources can be stretched a
lot further when only a single dose has to be administered.
For individuals, too, the Johnson & Johnson vaccine has benefits. As a
one-and-done injection, it's more convenient. It also has a lower rate of
adverse events than Moderna's. You can't compare results of these trials too
precisely, but there are indications of a striking difference. About 2
percent of those who got the Johnson & Johnson vaccine recorded having
reactions, such as fatigue, muscle aches, and fever, that were severe enough
to interfere with daily activities. For those getting their second injection
of Moderna, that rate was higher than 15 percent. People who are on the
fence about getting vaccinated may find that this difference tips the scales
in favor of getting a shot. Others who have doubts about the newness of the
mRNA technology in the Pfizer and Moderna vaccines may appreciate the fact
that Johnson & Johnson's approach has already been deployed in the company's
Ebola vaccine, which got full drug approval in Europe last year.
Given these concerns, there's some danger in the message-however well
intentioned-that the COVID-19 vaccines are all the same by any measure, or
that they're perfect wards against severe disease. Vaccination is a
public-health imperative, and going full tilt to promote uptake supports the
common good. But it's a personal health decision too. People want to protect
themselves and those close to them, and they are likely to care about
outcomes other than hospitalization and death, no matter what anyone says
now.
Still, raising these concerns in public can be fraught. In response to an
inquiry about her data table, Gandhi affirmed the importance of looking at
severe-disease outcomes and noted that "careful, collegial and collaborative
scientific discourse on the vaccines is imperative moving forward to help us
get through the pandemic." Topol pointed out that he has emphasized the
vaccines' measured efficacy against symptomatic disease many times before,
so any isolated reference to his table "takes that particular post out of
context." Jha wrote in an email that he stands by the message of his
original tweet, and notes that COVID-19 hospitalizations and deaths are so
rare among the people vaccinated in these trials, to quibble over
differences is akin to "counting how many angels are dancing on the head of
the pin."
I can see why this might seem like quibbling, but I just don't think it's a
trivial matter. It would be different if I thought the effectiveness of
every one of those six vaccines against hospitalizations and death would
really end up being close to 100 percent-or if I bought into the idea, now
widespread, that they have already been shown to "nearly" or "effectively"
eliminate these outcomes. There is very good reason to be encouraged by the
data, but to say right now that people who have been vaccinated face zero
risk of serious outcomes-that, for them, COVID-19 is no more dangerous than
the common cold-is sure to influence behavior. Imagine how people in
high-risk groups would feel about going to the movies, or how their
employers would feel about putting resources into workplace safety, if we
all assumed that vaccines confer perfect protection against hospitalization
or death. Now imagine how the same people and employers would feel knowing
they were 85 percent protected.
Nor is there any reason to believe that the public or the personal interest
will be served by hype. People who think the vaccines provide ironclad
protection may lose trust in experts if reality falls short. Trust in
coronavirus-vaccine information is already a problem, and could sink even
lower. Activists who are opposed to vaccination may end up turning experts'
"super-pumped" promises against them.
"The idea that people can't handle nuance," Jha tweeted at the end of
February, "it's paternalistic. And untrue." I couldn't agree more. The
principle of treating people like adults is fundamental. We don't need to
exaggerate. Talking about the trade-offs between different medicines and
vaccines is often complicated, but we do it all the time-and we can do it
with COVID-19 vaccines too.
