The parents of four Maryland teens are suing the Baltimore County Police Department, the county's school board, and several individual officers and school district employees after an AI gun detection system falsely flagged the teens for having a firearm on school grounds.
The incident, which happened last October, drew an armed response to Kenwood High School. One student was allegedly held at gunpoint and placed in handcuffs while police searched in vain for the gun he supposedly had. As it turns out, the AI system was mistaken, and what it reported as a gun was in actuality nothing more than a bag of Cool Ranch Doritos.
As we reported last year, the company in charge of the AI system says a human should have reviewed the alert, but says otherwise the system functioned as it was supposed to. And Baltimore County Public Schools Superintendent Myriam Rogers also said the system worked as planned, telling a local TV station that "the program is based on human verification and in this case the program did what it was supposed to do which was to signal an alert and for humans to take a look to find out if there was cause for concern in that moment."
When I hear the phrase "human verification," I picture someone in front of a computer monitor looking at the same image that AI has seen. I don't imagine it involving multiple armed police officers who are responding as if the AI's alert has already been proven accurate.
“This case is about the people who turned that machine error into a traumatic police takedown of four teenagers at their own school: the police officers who, guns drawn, forced the students to their knees on the concrete, handcuffed them and searched them; the school officials who unleashed this technology on their own students with no standards or protocols to protect against false Alerts; and the company that designed, sold and monitored the faulty technology,” plaintiffs Jerrie Davis and Lamont Davis say in the 31-page complaint.
Upon arriving at the school, the officers exited their vehicles with guns drawn at the four students and without any explanation why they were suddenly being detained. Only after the officers’ search of the students revealed no weapons did an officer mention they had been alerted by Omnilert.
“The officers on the scene knew [Omnilert Gun Detect] was unreliable,” the parents say in the complaint. “One told the students that ‘AI is not the best.’ Another muttered ‘stupid AI cameras’ and ‘f***ing Omnilerts.’ Yet officers relied on the alert to initiate this painful encounter with the students.”
The parents say one officer suggested in parting that, had the students not immediately complied and gone to their knees, “that would have been a real problem.”
According to the parents' complaint, but the responding officers and Kenwood High School principal Catherine Smith made light of the incident afterward, with Smith alerting staff via a text that included four laughing crying emojis. The lawsuit also alleges that neither Smith nor any other school district employee has offered any apologies to the students.
We all want our kids to be safe at school, but I'm not convinced that outsourcing that job to AI is the best way to do it. Honestly, though, whether this snafu was the result of computer error, human error, or a combination of the two, is it really that difficult to recognize that these students were freaked out when police pulled up, drew their guns, and started patting them down for what turned out to be an after-school snack?
How hard is it to apologize for that mistake? Apparently too difficult for school district officials.
I can't help but wonder if a mea culpa might have been enough to stop this lawsuit from ever being filed. Regardless, it's too late for that now.
There's a larger issue here, too. The same people who insist that guns are too dangerous to be trusted in the hands of ordinary citizens should be just as concerned about a technology that leads to government officials pointing guns at people who haven't done anything wrong. These four teenagers deserved better than a terrifying encounter followed by a few laughing emojis, and they deserve some accountability for what happened to them. As far as I can tell, Maryland's gun control lobby and anti-gun politicians have been conspicuously silent about what happened to these kids. Apparently, guns in the hands of government employees aren't nearly as concerning as guns in the hands of law-abiding citizens.