Four Black students at Kenwood High School in Baltimore County were handcuffed and detained at gunpoint by police on October 20, 2025, after the school's Omnilert AI gun detection system flagged a Cool Ranch Doritos bag as a potential firearm. Although school safety officials reviewed the camera footage linked to the alert and canceled it within two minutes—recognizing the object as a chip bag and notifying recipients, including the Baltimore County Police Department, that no response was needed—Principal Kate Smith allegedly failed to see the cancellation. Approximately 15 minutes later, she texted a school resource officer to deploy police, acknowledging the object was "maybe not" a gun. At least six patrol cars responded with guns drawn; officers ordered the teens to their knees, handcuffed them, and searched them without explaining the suspicion. The Office of the Inspector General (OIG) for Education found no bias in Omnilert's algorithm but concluded the school's "delegation of authority" process failed, noting Smith had never been trained on the system and that over 99 percent of Omnilert alerts in the district were false positives. The OIG recommended biannual training, reducing alert recipients, and using the Omnilert app to prevent missed cancellations. A lawsuit filed by the students and the Lawyers' Committee for Civil Rights Under Law alleges constitutional violations, deliberate indifference, and that Omnilert markets a defective product claiming "near-zero false positives" without enforcing adequate human review protocols. The plaintiffs seek an injunction halting Omnilert's use until proper safeguards and training are established.
Read original
arstechnica/ai