"tests prove X" claim can't be falsified by showing not-X, court (wrongly) rules

Dyson, Inc. v. SharkNinja Operating LLC, No. 14-cv-09442 (N.D. Ill. Apr. 26, 2017)

Dyson and Shark compete in the market for vacuum cleaners. When Shark began running an infomercial for its competing vacuum in September 2014, Dyson’s margin on its DC65 vacuum fell from $300 per vacuum to less than $100 per vacuum. The September 2014 infomercial claimed “more suction and deep cleans carpets better than Dyson’s best vacuum,” with a super, “Shark NV650 v. Dyson DC65 based on ASTM F558 measured at the hose and ASTM F608 embedded dirt.” There were also similar print ads and short TV ads. The packaging and, at some point, Shark’s website also made similar claims.

The CEO said, in the infomercial: “I have the independent lab tests to back it up. We asked independent testing facilities to conduct the one and only industry-recognized test of carpet cleaning, and we went head to head with Dyson’s best. Both vacuums were tested on four of the most commonly owned carpet types in America. And when all was said and done, the independent lab tests proved without question that our new Shark Rotator Powered Lift-Away deep cleans carpets better than Dyson’s best $600 vacuum.” The super read, “Independent LAB TESTS PROVE . . . Dramatization footage of ASTM F608 embedded dirt (NV650 in carpet/low pile mode) Shark NV650 vs. Dyson DC65.” In October 2014, a revised infomercial said similar things until the statements were removed in August 2015.

The general principles that come out of this: Dyson argued that Shark’s advertisements referencing “independent” tests were false, because the tests were performed by Intertek, an entity that was not independent of Shark because Shark paid it $1 million per year for various tests. The court evaluated falsity by what “a linguistically competent” person would think independent means “according to ordinary usage.” That would mean “free from outside control” and “not beholden to.” Dyson didn’t submit sufficient evidence to go to a jury. “[T]he mere fact that a customer pays for a service does not mean the service provider is controlled by the customer. In considering independence, the issue is not whether Intertek is paid but whether Shark is such an important customer as to make up a material portion of Intertek’s business.” Dyson didn’t put forth evidence on this question. Contacts between Intertek and Shark, including an Intertek employee’s email to Shark about a Dyson ad and discussions about the proper settings to test the vacuums, didn’t show that Shark controlled Intertek clearly enough to avoid a jury. Thus, Dyson couldn’t get summary judgment on its claim.

The court rejected Dyson’s claim based on the ad statement that Shark’s vacuum deep cleaned carpets better than “Dyson’s best” vacuum. Though a new vacuum called the Ball Multi-Floor supposedly became Dyson’s “best” in April 2015, Dyson didn’t provide evidence of its bestness, and in any event Shark always disclosed via an asterisk the Dyson model on which it was basing its comparison, not the Ball Multi-Floor. “[T]he court will not ignore the portion of Shark’s ad that explicitly states that DC65 is the Dyson vacuum to which Shark was referring.” Summary judgment for Shark.

Dyson argued that Shark had no independent tests to prove the superiority of its vacuum from July 8 to August 12, 2014, because Shark did not receive the final report establishing that its vacuum was superior until the latter date. The court agreed that Dyson had shown falsity: Dyson’s packages made the superiority claim, and Shark didn’t show that an earlier version of the report supported its claim. Hark argued that the information on the packaging could not have affected the purchasing decision of customers because, at that time, the vacuums were available for sale only on the website, so customers would not have seen the claim on the box until after they had made the purchase decision. “A reasonable jury could conclude that a statement on a box that the customer could not see until after he purchased the vacuum was not material to the purchase decision.” (Dyson made the same argument about stale claims of its own on Dyson packages sold through its website.)

As to the “tests prove” claims, Dyson argued that even if it failed to show that Shark’s test is invalid, Dyson could still win by putting forth its own tests showing that Shark was no better than the Dyson. Shark rejoined that the only way to prove a claim that “tests prove x” was false was by showing the tests do not prove x. If Dyson were right, its claims would survive summary judgment. The Seventh Circuit rule is the standard one: “If the challenged advertisement makes implicit or explicit references to tests, the plaintiff may satisfy its burden by showing that those tests do not prove the proposition; otherwise, the plaintiff must offer affirmative proof that the advertisement is false.”

Surprisingly to me, the court agreed with Shark. “[A] claim that the ‘test proves x’ is literally false only if the test does not (reliably) prove x.” Shark’s valid independent tests supported its statements. “That Dyson conducted other tests that reached a different conclusion does not make Shark’s statements about its tests false.”

RT: What if VW advertised that emission tests proved its cars met emissions standards? Wouldn’t we say that those tests didn’t “prove” X, which could be falsified by other evidence? In my opinion, there are two claims here—the tests prove claim, which adds credibility to X on its own, and X. I have always understood that the statement “tests prove X” claims both that tests prove X and that X is, in fact, true; otherwise why would the tests be relevant? Separately, this result seems problematic from a pure statistical perspective. Suppose Shark had a valid test that showed superiority at the 95% confidence level, but Dyson had 19 valid tests that showed no superiority at the 95% confidence level. A 95% confidence level suggests that the test will only be wrong one out of twenty times; with that evidence, a jury should be able to conclude that Shark’s test, though valid, did not “prove” Shark’s claim.

Nonetheless, the court granted Shark’s motion for summary judgment as to claims where this was the key argument.

Shark argued that Dyson had unclean hands due to Dyson’s continued use of the phrase “Twice the Suction” to describe its vacuums after that claim became stale. But this conduct didn’t arise out of the same transaction from which this case arises, so summary judgment for Dyson on this defense was merited. Also, “[t]he conduct about which Shark complains in its affirmative defense is conduct the parties are already fighting about in another lawsuit. To apply the unclean hands doctrine here (or there) would leave the alleged wrongs without remedy.”