The Tampa Bay Times has a pretty good thing going with their fact-checking site Politifact.com, and their so-called Truth-O-Meter. The “fact checks” on the site tend to follow a reasonably intelligent pattern.

• Present a disputed claim, and the context in which it occurs
• attempt to find the basis or source for the claim
• attempt to determine if the information provided by the source is valid, correct, or relevent
• render a conclusion.

When Politifact.com renders their conclusion, they use the following scale.

• True – The statement is accurate and there’s nothing significant missing.
• Mostly True – The statement is accurate but needs clarification or additional information.
• Half True – The statement is partially accurate but leaves out important details or takes things out of context.
• Mostly False – The statement contains some element of truth but ignores critical facts that would give a different impression.
• False – The statement is not accurate.
• Pants on Fire – The statement is not accurate and makes a ridiculous claim.

Earlier this month, they took on Gun Owners of America Executive Director Larry Pratt’s claims on the reliability of so-called “smart guns.”

The debate over “smart guns” doesn’t always turn up accurate talking points.

Exhibit A: Sarah Palin telling the National Rifle Association crowd about a (nonexistent) federal plan to make gun owners wear “special bracelets that would identify you as a gun owner.”

Exhibit B: MSNBC host Chris Hayes’ recent on-air shouting match with Larry Pratt, executive director of the lobbying group Gun Owners of America.

Smart guns — also known as “child-proof guns” and “personalized guns” — are designed to work solely for authorized users through features like fingerprint recognition or electronic sensors. The hope is they will reduce deaths by suicide, accidental shootings or someone stealing a law enforcement officer’s weapon. They are not sold in the United States, though two dealers tried before backing down amid intense pressure from activists.

Hayes supports the technology, but Pratt and other gun-rights advocates are skeptical. Pratt questions the guns’ reliability, saying smart guns are “only 80 percent effective.” When Hayes pressed Pratt for the source of his statistic, Pratt cited the New Jersey Institute of Technology.

“Twenty percent of the time it won’t work,” Pratt said. “And you’re asking people to put their lives in the hands of a product like that?”

Later, Pratt used the figure again. “Is it okay to put on the market a car that 20 percent of the time explodes on you and causes you harm or death?”

Since Pratt tripled down on his statistic, PunditFact wanted to check it out.

A claim asserting that “smart guns” (and we’ll drop the quotes for the rest of the article) have a 20-percent failure rate is an alarming claim worth checking out… and how Politifact.com chooses to check out that claim is just as interesting as the claim itself.

As Pratt was citing the New Jersey Institute of Technology (NJIT) as his source, Politifact attempted to trace the claim to the source.

Tracing the source of the claim

Pratt pointed to a 2003 column in Popular Mechanics magazine headlined “‘Smart’ Guns: Dumb Idea!”

The column was inspired by a 2002 New Jersey law that requires all guns sold in the state to have “smart” technology within three years of the attorney general verifying the first smart gun was sold in the United States. Fear among gun rights activists that more government mandates may come when this happens drive their opposition to smart guns.

In his column 11 years ago, writer Cliff Gromer said law enforcement officers are exempt from the law, which proves the technology isn’t solid enough for the mainstream gun market.

“According to the New Jersey Institute of Technology, which used government grants to study personalized handgun technology, fingerprint recognition systems work only 80 percent of the time,” Gromer wrote. “But the New Jersey law goes into effect regardless of whether the guns are 100 percent — or 80 percent — reliable.”

Pratt indirectly cited a 2003 Popular Mechanics, which cited the NJIT’s statement that “fingerprint recognition systems work only 80 percent of the time.” The data is dated, and cites just one (fingerprint recognition, a subset of biometrics) of the two prevalent forms (the other being RFID-equipped) of smart guns under development. Using Politifact’s own established rating scale, Pratt’s statement should fall somewhere in the middle of their rating scale:

• Mostly True – The statement is accurate but needs clarification or additional information.
• Half True – The statement is partially accurate but leaves out important details or takes things out of context.
• Mostly False – The statement contains some element of truth but ignores critical facts that would give a different impression.

The 80-percent figure is what NJIT mentioned, but the data is old and focused on only one part of the two most prevalent smart gun development paths. A fair rating would be to give Pratt’s claim a “Half True” rating.

Politifact, instead, decided that they wanted to give Pratt a much harsher—and in my opinion, completely unjustifiable— rating of False.

Here’s how they tortured their own methodology to manufacture that conclusion.

Article is not the smoking gun
You may have read over it, but the 80 percent figure Gromer cited talks about fingerprint recognition systems. Not whether a smart gun will work.
That’s important because while some smart guns use fingerprint recognition systems (which work like the new unlocking method for iPhones), not all do.

The German-made Armatix iP1 .22 caliber pistol was expected to be the first smart gun sold in the United States this year until a California gun store owner scrapped plans to sell it after facing aggressive backlash. It happened again with a Maryland dealer.

The Armatix does not have a fingerprint recognition system. Instead, the gun works with a black stopwatch and PIN code that emits radio signals to the gun to make it active. The signal is akin to what’s emitted from those bulky tags on mall merchandise that aren’t triggered until taken out of the store. Other models work with similar chips implanted in everyday accessories, such as bracelets, pins and rings, and some are implanted in the body.

If the badge is too far away from the microchip in the gun, it will stop working. But that means the technology works, not that it failed.

We mentioned briefly before that the article Pratt cited discussed only fingerprint recognition system, a subset of biometrics. In terms of smart gun development, researchers seem primarily focused on print recognition (finger or palm) and hand or grip geometry/pressure-based, focusing on the individual characteristics of how a specific shooter holds a handgun [It is my opinion that this is likely going to prove to be a dead-end development path for a wide range of reasons, starting with the simple fact that people wearing gloves would be unable to use print-based technologies, and the physiological stressors of a life of death situation are going to create radically different grip geometries and pressures than practice shooting, rendering a technology that is deadly deficient by design].

The most stunning claim that Politifact makes here about the Armatix is that a critical design flaw in the creation of the iP1/iW1 gun and watch combination is not a failure, but proof that the system “works.”

Good trainers and many forms of shooting competition note the unassailable fact there are many variables in real world scenarios that force shooters to alter or change their grips. While most of us tend to practice shooting using the modern technique of the pistol (“two-handed grip on the pistol and brings the weapon to eye level, so that the sights may be used to aim at one’s target”), there are many instances where a shooter might choose (or is forced to) fire one-handed, and the shooter may need to shift the gun back and forth between dominate and non-dominant hands due to injury, to take advantage of available cover, etc.

The iP1/iW1 gun and watch combination utterly fails to take this into account. The gun will only fire if the PIN code is entered correctly, and the watch is within ten inches of the gun.

We’ll give the Armatix system the benefit of the doubt and assume that you will have the iW1 watch on when you need to use your iP1 pistol.

We will further assume that the user remembers to activate and reactivate the PIN code as needed, which can be authorized for a minimum of one hour and a maximum of eight hours at a time. If the selected time expires and the user must activate the watch’s PIN code immediately before needing to use it, this is a critical design flaw for all humans not equipped with extra-sensory perception and who cannot see into the future.

We’ll also (necessarily) assume that the reader has adjusted to the requirement of the system to wear the iW1 watch on the dominant shooting hand. This a reversal of the common tendency for right-handed people to wear watches and their left wrist, and for left-handed people to wear their watches on their right hands.

If all of these very specific conditions are met and the Armatix-equipped person need to use their iP1 handgun in self-defense, it should (theoretically) work as long as the shooter is using the modern technique (two hands on the gun) or is using the dominate hand. The moment the shooter is required to shift to shooting from the arm that doesn’t have the watch—the shooter is attempting to ward off and attacker, is injured, is attempting to shield a family member—the system will not fire. It will, in fact, fail to fire 100% of the time.

Armatix and Politifact consider this to be an example of the technology working.  One can only assume that Politifact considers a car that only drives forward and turns only to the right due to poor design is also an example of “technology working,” instead of completely incompetent and dangerous design.

Politifact then goes on to discuss the “dynamic grip recognition” system that I assure you will fail at an unacceptable high rate due the great physiological differences between relatively stress-free target shooting and life-or-death combat (the Human Factors Research Group would have a field-day with this idiocy), before moving on to the one real inarguable fact in their debunking, which is that the NJIT research is too dated to be of any use.

Now, here comes what I consider the fun part. Politifact brings in college professors that have no credentials at all showing that they know anything at all about actual firearms design or usage in a classic appeal to authority fallacy, where the “experts” aren’t even experts in the field being discussed.

If not 80 percent reliable, then what?

Finding a reliability rate for smart guns is an “almost unanswerable question,” said Stephen Teret, a pro-smart gun Johns Hopkins professor and director of the university’s Center for Law and the Public’s Health. “Because there’s a wide range of quality in existing guns, some are better, some are worse. So when you are trying to compute a rate of reliability, are you using one gun or various categories?”

Robert Spitzer, a SUNY Cortland political science professor and author of The Politics of Gun Control (whom Hayes brought in after Pratt to talk about the issue), said in order for the Armatix iP1 gun to be sold in California, it had to pass a reliability test.

“The standard was a 99 percent successful fire rate, which it met (fire 600 times with 6 or fewer failed discharges),” Spitzer said.

Others stressed that all guns are prone to failure at some point. The mechanical failure rate is often given as 1 in 1,000, and a military weapon’s failure rate is closer to 1 in 10,000, Sebastian said.

“Misfire is embedded in our language,” he said.

Stephen Teret‘s area of expertise is as a gun control supporter, working in Michael Bloomberg’s School of Public Health at Johns Hopkins. He is roughly as qualified to talk about smart guns as I am to talk about mammography (though I suspect I have more amateur mammography experience than he has “hands on”experience with actual firearms of any type).

Likewise, Robert Spitzer is a college political science professor and supporter of gun control. He has no background in engineering of any sort, much less in electronics, firearms, or the combination of the two. He is only brought in as an “authority” to make a very suspect claim.

Robert Spitzer, a SUNY Cortland political science professor and author of The Politics of Gun Control (whom Hayes brought in after Pratt to talk about the issue), said in order for the Armatix iP1 gun to be sold in California, it had to pass a reliability test.
“The standard was a 99 percent successful fire rate, which it met (fire 600 times with 6 or fewer failed discharges),” Spitzer said.

Oh really? Spitzer’s claim to use Politifact’s own rating scale, is only “mostly true” to “half true.”

While the iP1/iw1 combination did pass the very carefully controlled reliability test (which does not account in the slightest for the real world factors previously discussed) in California, it has encountered problems in attempting to qualify for Maryland’s approved handgun roster, according to a 2013 National Institute of Justice report A Review of Gun Safety Technologies, which noted on page 63:

NIJ and SSBT Center staff observed a test firing of the Smart System at the Maryland State Police Forensic Science Laboratory in February 2013, which occurred as a part of the Maryland regulatory process. Multiple pistol units were present for visual inspection as shown in Figure 13. Per the testing protocol, one example firearm discharged the first 20 rounds without malfunction, although some user difficulties were observed with the initial synchronization between the gun and the wrist-worn device which were resolved.

Our Conclusion

The evidence that Politifact used to rate Pratt’s claim as “false” appears itself more subjective that the Pratt’s original claim.

While Pratt’s claim is based upon old data, and he was only referencing subset of smart gun technologies, his claim deserves no worse than a “half true” rating.

On The Other Hand…

Politifact’s bizarre attempt to claim that a serious design flaw by Aramatix is proof that the technology works is absurd. A car without a reverse gear that only turns in one direction is a serious design flaw, just as a gun that can only be fired by a designated hand and fails to work for the other hand of an authorized user is a serious design flaw.

Politifact’s attempt to insert two non-experts with overt biases into the conversation, transparently using “appeal to authority” fallacy, is sophomoric.

Their attempt to assert that carefully controlled conditions in a California test proves that the gun is reliable, when the gun failed to synchronize properly in similar tests in Maryland is an example of cherry-picking data, made only worse by the fact that neither laboratory test comes close to matching real world conditions.

If Politifact wants to fact-check something truly interesting about smart guns, they should check out this claim by New Jersey Assemblyman Michael Patrick Carroll:

He ridiculed manufacturer Armatix’s claim that it could determine, with 90 percent accuracy, whether a gun was being held by a person wearing a watch meant to pair with the firearm. And he took aim at its claimed ability to only fire when pointed at its target.

“When I’m talking about targets, I’m talking about someone who’s about to rape my wife,” Carroll said. “And I want to be sure that the gun goes off when I’m firing at that particular target.”

Is it true that Aramatix itself only claims a 90-percent success rate in their own tests?

That would be interesting if true, as it would seem to suggest that it only works 90-percent of the time single-handed in the authorized hand, while it works 0% of the time in the authorized user’s opposite hand.

I’ve never claimed to be very good at mathematics. Perhaps Politifact can tell use whether “90%, of 50% of the time,” is more or less than the 80% figure what Pratt cited.