The injured teenage survivor of a January 2025 shooting at a Nashville, Tennessee high school recently sued the manufacturer of an “AI gun detection” system that failed to detect the handgun that left two dead, including the shooter.
According to the lawsuit, which was filed in Davidson County court last month, the security company Omnilert either knew or should have known that there were “significant operational limitations in its gun detection system that could result in detection failures during actual emergencies, including limitations based on camera placement, proximity of the weapon to camera sensors, camera angle, lighting, and weapon visibility.”
Omnilert cofounder Ara Bagdasarian declined Ars’ invitation to answer questions about the lawsuit. System Integrations, the other defendant in the case, which resold the Omnilert system, also did not respond to Ars’ request for comment.
I am so tired of AI being shoved into everything and then people surprised when it doesn’t work. There’s no AI I think that could have detected a small firearm easily concealed. Hell as it is with legal concealed carry you can’t tell who is legally carrying as it is even with some of the most observant eyes watching.
The idea with these kinds of systems are meant to allow early warning when possible.
No system is going to be 100%.
Edit: I get the downvotes, but there are people/companies that were/are developing such systems with a genuine intent to make things better. I know, because I was one of them.
The cheap system I have with a Google Coral and FOSS software is pretty good about detecting people and dogs from my camera streams. Sometimes it detects my one dog as a small bear if I haven’t cut his hair recently.
Having such systems as a later if defense is good. As the only defense, not so much.
I’m guessing you’re using Frigate?
Agreed. The system I had developed was built explicitly as a human-in-the-loop system. It never made any decisions on its own. It was just a tool to enable the existing security staff to have better visibility. That’s it.
You can make whatever argument you want about viability and efficacy. The only point I’m making is that our system was just an additional tool for security to use; not the only one.
Yup. I wasn’t disagreeing so much as pointing out a common failing with AI adoption in general: the number of cases where it’s being implement as a replacement for existing functional systems or humans rather than an augment to them. The “additional tool” aspect is 100% my preferred use case (where it functionally makes sense), but many are seeing it as a human-replacement that feeds into their desire for control/subservience, cost-reduction, or rent-seeking.
Local-AI actually has a lot of useful cases, and AI vision is something that in many forms has been around for awhile and is generally fairly effective. It’s great as a tool for indexing or enriching visual data that would otherwise at best a slog and at worst improbable for a human. Surveillance video is especially good with this. I see it as an addition to motion detection. Instead of needing to go through hours of footage of video, you only need to go through stuff where movement was detected, and from that you could further search for “frames tagged as having a person/bear/vehicle/whatever”. The thing is, I’m not depending on it to protect me from running into a bear when I go out out my trash (although Frigate can do alerting), but I could use it to follow how the bear moved between cameras/zones through my property and possibly use that to better bear-proof the premesi
In this particular case, it might be somewhat useful as an early warning if it can actively detect human+firearm, but there were obviously going to be either a number of false positives or negatives that make it less valuable as such. It could still be useful in reconstruction or actively following some incidents, but we need to keep humans in the loop too.
AI isn’t bad/evil or good when used correctly and with knowledge of the limitations in a given use-case. Trusting AI to replace humans/judgement is often very bad, and massive datacenters are obviously a major source of concern, but the issue is again in the use not the overall technology. Just like nuclear fission can produce huge amounts of power to either keep the lights on or blow stuff up, AI can help sort through your album if 5000 photos to find that one shot in a jiffy… or it can be used identify, track and you for a surveillance-state, and false positives in the latter case can make for a very bad day.
AI in this context is useless though, you could paint marker “not a gun” on the side of a gun and guess what would happen.
It has some uses, but 95% of what is being used for and 100% of the data centers aren’t it.
It would flag it as a gun. How do I know? I worked on and developed a similar system at one point. It worked extremely well. We weren’t an American company and ultimately covid killed us (it was US American orgs that were the most interested in our stuff).
Do you think LLMs are being used for this sort of thing? Putting aside the sheer technical mountain of a hurdle that slapping an LLM vision model on top of dozens and dozens of real-time camera streams, the hardware requirements would put the company out of business before they made their first sale.
Computer vision models, which are NOT LLMs, have been around for quite a while now and are very good at doing one thing and one thing only. And they’ll do it well for a miniscule fraction of what it takes to run an LLM.
No, datacentres are not being used for real-time gun detection. The company might have other kinds of infrastructure located in a DC, but not the main video processing hardware.
Yes. It took all of five seconds to find out too.
You’ve already been wrong once, care to try for two?
Didn’t I just say that slapping an LLM vision model on to dozens of camera streams would be a near impossible technical hurdle?
I never said vLLM models don’t exist. I said they’re impractical for this use case.
Haven’t been wrong yet. You on the other hand…
There are several examples of exactly what I said, contradicting your repeated claim. Since I don’t want to talk to someone with the conversational ability of Donald Trump demanding things be true in spite of evidence they’re not im going to be blocking you now. Have a nice day.
No one is denying the existence of vision based LLM models. The issue is performance. It takes in the order of double (or even triple) digit seconds to process an image through an LLM. Even if it took a single second to process an image using decent server-grade hardware (which starts at about $10k per card), that’s way too much and still not fast enough.
On just 10 cameras at a facility it would require north of $100k on just GPUs alone.
Whereas a specialized computer vision model could process several dozen camera streams, in real-time, on just one of those $10k cards.
An LLM would process an image in 10 seconds (generous) whereas a computer vision model operates in the milliseconds. We’re talking about a 1000x difference in required processing power.
That’s why you’re wrong and have zero clue what you’re talking about.
You’re arguing that that family uses a fully loaded semi-trailer to go 200m to the local park. It’s a clueless and asinine argument.
Using a LLM for detecting a specific object on an image is possible but stupid: if your object is always the same (like in this case) it’s several orders of magnitude cheaper to train once on that specific object then use the computer vision model running directly on the local server that’s recording the video.
Otherwise:
Use cases for LLM-based image recognition is if the object changes at every request or it’s ultra specific with brands and colors
It isn’t the same though. A large gauge shotgun and a small gauge pistol are pretty different looking. Compare those to a .22 rifle with a scope, and those to a decked out ar15. That’s a lot of different always the sames. What if it’s a revolver? Or has a folded stock? Or a sawed off stock? Will it recognize a derringer or a mac10 with a large capacity mag as guns?
We can because they make us dead. We have valid reason to fear them which is a great motivator for most species to learn to recognize the danger. You’d still recognize a ring gun as a gun, without getting specifically trained to do so a machine will identify it as jewelry.
You seem to think that computer vision models can only be trained on a single thing. You simply train your modem on as many object types as you want it to be aware of. That’s it.
so, train the computer vision model for a gun and train again for a shotgun. Run the two detection models at the same time.
Your approach is the typical “but if you really want you can use an atomic bomb to kill mosquitoes” - yes, you could do that, but nobody is paying $1 mil/year in inference costs (+some expensively licensed software to wrap around that) when it can be done locally with a $300 GPU (+ some expensively licensed software to wrap around that)
I gave a lot more than two examples and it was hardly exhaustive.