Events Payload

Object detection event payload

Overview

Traffic Monitor object detection events are generated by the connected sensors by performing object detection to identify instances of roadway users such as cars, bikes, pedestrians, and more. Additional data and metadata may also be added via other sensors and processes to create an event payload.

Hardware and Software

The camera is the primary detection method, powered by Frigate NVR. Events are created by optical camera observation and machine learning-powered object detectors inference the frames to label available objects.

Future feature: The radar may also generate object detection events. This is particularly useful for nighttime and low-light conditions or even deployments that do not utilize a camera. See Radar Payload for more information on radar readings.

Frigate MQTT Incoming Payload

Events are recorded via the Frigate MQTT integration on the frigate/events topic containing type: "end",which indicates a complete capture event. See Frigate documentation for a full list of available attributes.

Events Database

The following attributes are captured, by default, into tmdb.events.sqlite:

Attribute	Description	SQLite data type	Valid values
id	UUID for object detection. Will be generated by `source` Frigate-generated from MQTT event `end`. Radar-generated from flow.	TEXT, PRIMARY KEY	Frigate: Unix timestamp in seconds concatenated to a hyphen and randomly-generated 6-alphanumeric value; e.g. "1721144705.617111-fg3luy" Radar: Unix timestamp in seconds concatenated to a hyphen and randomly-generated 8-alphanumeric value concatenated with a hypen r '-r'; e.g. "1721144705.617111-fg3luy4x-r"
camera	Name of camera for object detection (defined in Frigate config). Frigate-generated from MQTT event `end`.	TEXT	Free-text
label	Object label assigned by Frigate. Frigate-generated from MQTT event `end`.	TEXT	Assigned by model, based on Frigate available objects
sub_label	Additional informatoin assigned to Frigate event. Frigate-generated from MQTT event `end`.	TEXT	Assigned via Frigate HTTP API
top_score	Model inference score for object label. This is the highest score as object moved through field of view. Frigate-generated from MQTT event `end`.	REAL	0-1 value
frame_time	Unix timestamp in seconds for when the object was optimally identified by Frigate for the field of view. Frigate-generated from MQTT event `end`.	REAL	Unix timestamp in Seconds
start_time	Unix timestamp in seconds for when the object first entered the field of view. Frigate-generated from MQTT event `end`.	REAL	Unix timestamp in Seconds
end_time	Unix timestamp in seconds for when the object exited the field of view. Frigate-generated from MQTT event `end`.	REAL	Unix timestamp in Seconds
entered_zones	JSON array list of zones in order the object entered each zone. Specified zones are used for various calculations. Frigate-generated from MQTT event `end`.	TEXT	Free-text via Frigate; expected: - "zone_capture" - region for counting objects - "zone_radar" - radar detection field of view (FOV) - "zone_near" - area closest to radar, for determining visual direction - "zone_far" - area furthest from radar, for determining visual direction
score	Model inference score for the object to initiate tracking. Computed score as object moves through field of view. Frigate-generated from MQTT event `end`.	REAL	0-1 value
area	Width*height of the bounding box for the detected object Frigate-generated from MQTT event `end`.	REAL	0-24000000
ratio	Width/height of the bounding box for the detected object; e.g. 0.5 is tall (twice as high as wide box) Frigate-generated from MQTT event `end`.	REAL	0-24000000
motionless_count	Number of frames the object has been motionless Frigate-generated from MQTT event `end`.	REAL	Integer count; e.g. 0
position_changes	Number of times the object has changed position Frigate-generated from MQTT event `end`.	REAL	Integer count; e.g. 2
attributes	Attributes with top score that have been identified on the object at any point Frigate-generated from MQTT event `end`.	TEXT, JSON	JSON object with key:value pairs; e.g. {"face": 0.86}
direction_calc	Assigned object moving direction relative to device placement; i.e. "outbound" is moving away from device.	TEXT	"outbound" or "inbound"
speed_calc	Assigned speed/velocity calculated for entire time object was in the camera's field of view.	REAL	Positive, Negative corresponding to inbound and outbound direction, respectively
provenance	Source(s) of event detection. List any sensor on the device that captured or created this event. The first item in the array is considered the primary source.	TEXT, JSON	JSON Array with every sensor that confirms the same event. e.g. Camera sensor: frigate, Radar sensor: radar
radarName	Radar sensor name that is associated (via config) with the camera during the event, regardless if event was confirmed by radar (see provenance).	TEXT	Free-text, defined from configs
deployment_id	Each ID represents a unique deployment configuration and/or location for the device. This acts a foreign key link to the `deployment` table, `id` column.	TEXT, FOREIGN KEY	`deployment`.`id` foreign key, may be null

Telemetry

HTTP Telemetry

ThingsBoard HTTP upload telemetry API requests are sent for each event as a single JSON payload containing:

• ts: frame_time * 1000 - to make it milliseconds

• values: {event:values}- contains all attributes in Events Database

MQTT Publications

The primary event are available on-device for downstream subscriptions (e.g. Home Assistant):

• Topic: tm/event

• Payload: Same as the Events Database

Additionally, there is a daily cumulative object count utilized for the on-device dashboard and connected displays:

• Topic: tm/events

• Payload: Daily cumulative counts of detected objects (resets at 0400 local time). Note, these can be adjusted in the Frigate config for Available Objects.

  • car

  • person

  • bicycle

  • motorcycle

  • bicycle_adj - bicycle plus motorcycle - adjust for eBikes, but this is addressed now by Frigate configs for each object and is unnecessary for most deployments

  • person_adj - person minus bicycle - Essentially represents "pedestrian count" since every bicycle will have [at least one] rider that is detected independently of the bike.

    • Scooters and other transit modes will likely only count as a person using the base model.

    • Cars never have a person identified inside of them with the base model.

  • dog

  • cat