Misrepresenting smallpox vaccine in pregnancy findings

Re: Evaluation of Preterm Births and Birth Defects in Liveborn Infants of US Military Women Who Received Smallpox Vaccine. Margaret A. K. Ryan, Gia R. Gumbs, Ava Marie S. Conlin, Carter J. Sevick, Isabel G. Jacobson, Katherine J. Snell, Christina N. Spooner, Tyler C. Smith, for the Department of Defense Birth and Infant Health Registry Team. Birth Defects Research (Part A) 82:533-539 (2008)

Last April I wrote a detailed critique in this blog of a paper by Ryan et al. on anthrax vaccinations in pregnancy and birth defects. In the paper I am discussing today, Ryan has studied birth defects in military servicewomen who received smallpox vaccine during pregnancy. In fact, there is likely significant overlap between the two groups of women Ryan studied, since both anthrax and smallpox vaccines have been given to all servicemembers deploying to Iraq, Afghanistan and Korea since 2003 (except during the period in 2004-2006 when Judge Emmett Sullivan stopped mandatory anthrax vaccines).

The findings of the two studies are, no surprise, very similar: women receiving smallpox vaccine during the first trimester of pregnancy had a rate of major birth defects in their offspring of 4.5%, while the offspring of women vaccinated post-pregnancy had a major birth defect rate of only 3.2%. The birth defect rate is therefore 40% higher if the mother was vaccinated during the first trimester.

The paper fails to explain significant problems with the military database, whose accuracy was previously studied by Ryan and discussed in my earlier blog. The sensitivity for identifying anthrax vaccinations in this database was less than 70%. How well does it correctly identify smallpox vaccinations?

Only 30 of 672 women vaccinated in the first trimester had infants born with major birth defects. So even though the birth defect rate was 40% higher, it did not achieve statistical significance. The authors then incorrectly concluded that, "smallpox vaccine, when inadvertently administered to pregnant women, is not associated with preterm delivery or birth defects in liveborn infants."

What would a true scientist have done with these results?

First, a true scientist would not have misrepresented the data. To be accurate, the paper should have said that a sizeable increase in birth defects was found, but given the numbers involved, did not reach statistical significance.

Second, the paper should have pointed out that a similar effect had been found for anthrax vaccination, in a larger but overlapping cohort of servicewomen, by three of the same authors. The two vaccines administered together probably caused confounding, making it impossible to determine how much each vaccine may have contributed to the birth defects (which were statistically significant at the 0.05% level for some of the anthrax findings).

Third, a true scientist would have expanded the sample size, to try and achieve statistical significance. This would have simply entailed using the same database over a longer duration. The paper was submitted in 2008, so there was time to obtain additional data on servicewomen vaccinated since 2004. Of course, then the result might no longer be used to support military vaccine policy.