Omni Responder

Perception by machine, judgment by human

Overview

Omni Responder is a computer vision tool that analyzes roadside camera footage to detect possible emergency situations and then notifies a human supervisor to investigate further.

The Premise

Transport authorities operate many thousands of roadside cameras. Although the feeds are often broadcast publicly, few are regularly monitored by a human. This raises concerns about both privacy and effectiveness. Our tool addresses these by observing and processing footage on the edge in a local AI system that runs on the same network as the cameras. The only data that leaves the system is strictly focused on emergency response.

The benefit is that human attention is focused where it matters most.

How It Works

A low-resource motion scan runs across the footage first, looking for moments where something changed and then stayed changed. These moments are then passed to a vision-language model, which describes the scene and rates the potential severity of the incident.

That description then fans out to three domain agents. One checks visible hazard indicators and returns isolation distances and protective-equipment levels. One drafts roadway closure and detour advisories. One assembles a dispatch brief of the kind an emergency call-taker would read. The result arrives as a live dashboard for human review and response.

Local by Design

All of the AI models run on a local device, the Nvidia DGX Spark. The footage is analyzed on that single machine, which could sit on the same network as the cameras, so no video is sent anywhere. Only information about a possible emergency is sent over the network.

Created along with Bilal Khan and Vishal Shah.

See it for yourself at GitHub.