Second Circuit summarily reverses bad Prevagen decision on statistical proof

FTC v. Quincy Bioscience Holding Co., 17-3745-cv(L) (2d. Cir. Feb. 21, 2019)

Quincy sold Prevagen dietary supplements, claiming (1) that the supplements improve memory and provide other cognitive benefits, (2) that these effects are clinically proven, and (3) that the products’ active ingredient “supplements” brain proteins that are lost with age. The FTC alleged that Quincy conducted a randomized, double-blind, placebo-controlled study that contradicted these representations and showed no statistically significant improvement in the memory and cognition of participants taking Prevagen over participants taking a placebo. Quincy subsequently “conducted more than 30 post hoc analyses of the results” of the study, and “the vast majority of these post hoc comparisons failed to show statistical significance.” While the study showed a “few positive findings on isolated tasks for small groups of the study population,” these findings allegedly did not “provide reliable evidence of a treatment effect.” The district court, in a poorly reasoned opinion, dismissedthe complaint because the post hoc analyses must have shown something. The court of appeals summarily reversed.

“The FTC has stated a plausible claim that Quincy’s representations about Prevagen are contradicted by the results of Quincy’s clinical trial and are thus materially deceptive in violation of the FTC Act and New York General Business Law.” For example, Quincy claimed broadly that in a clinical study “Prevagen improved memory for most subjects within 90 days,” yet this wasn’t true: the study “failed to show a statistically significant improvement in the treatment group over the placebo group on any of the nine computerized cognitive tasks.” Not only did this make plausible that it was deceptive to claim that “the majority of people” experience cognitive improvement from taking Prevagen, but the FTC also stated a claim that Quincy’s representations that this cognitive improvement is clinically supported are deceptive. Further, the FTC alleges that Quincy’s claim that the active ingredient in Prevagen, apoaequorin, “enters the human brain to supplement endogenous proteins that are lost during the natural process of aging” was false. Quincy’s “safety studies show that apoaequorin is rapidly digested in the stomach and broken down into amino acids and small peptides like any other dietary protein.” “Drawing reasonable inferences in favor of the FTC, as we must, the FTC plausibly alleged that Quincy’s representations about Prevagen’s active ingredient entering the brain are false.” The district court erred in dismissing the complaint and declining to exercise supplemental jurisdiction over New York’s claims.

The court doesn’t go into detail on the bad statistical understanding underlying the district court’s mistakes, but you can read the amicus brief I signed or let xkcd do it.