# Data Overview

## Background

The objective of the Traffic Monitor is to be used as a primary data collection tool to capture holistic roadway data. The architecture is privacy-focused, featuring onboard detection, processing, and local data storage for cloud-free capability. Data is processed, generated, and captured locally in real-time.

The system records granular, event-level records for all sensors including those for **detected objects** (e.g., bicycle, vehicle, pedestrian), **radar readings** (speed, magnitude, detected object velocity, etc.), **environmental measurements** (air quality, temp, pressure, humidity, particulate matter, etc.), **deployment metadata**, **system metrics**, and more including data from plug-ins and extensions.

### Data privacy

We believe in the power of [open data](https://en.wikipedia.org/wiki/Open_data). Anonymized data are crucial for useful social research and a public good. Our system (the Traffic Monitor database) prioritizes privacy by capturing as little [personally identifiable information](https://en.wikipedia.org/wiki/Personal_data) (PII) data as possible. With the default settings, our dataset is fully [de-identified](https://en.wikipedia.org/wiki/De-identification) so it is easier to share with mitigated risk to individuals and the public.

The Traffic Monitor may, via configuration file changes, capture PII such as license plates and images from persons, for example, via full roadway snapshots and clips from the camera. 

Be aware that when enabled, additional applications and sensors may be captured both in the [primary database](#database-structure), losing de-identified qualification, and elsewhere on the device storage; e.g. application-specific folders and databases that contain their own retention policies.

{% hint style="warning" %}
Applications, plugins, and sensors may also store data for their own operation data in their own structure and locations.
{% endhint %}

## Storage formats

The Traffic Monitor database utilizes unencrypted, open formats by default to make them portable and easy to share. 

* [SQLite](https://www.sqlite.org/) is the native database format and can be viewed with open source [DB Browser for SQLite](https://sqlitebrowser.org/).
* [JSON](https://en.wikipedia.org/wiki/JSON) - The [database](https://docs.trafficmonitor.ai/ui/database "mention") UI tab allows downloading data with specific filteres including by table, object(s), and/or time frames.

## Database structure 

The Traffic Monitor database contains all captured, combined, and processed data from sensors, including additional, optional applications. 

For tables related to sensors that are not installed, they will simply contain no records.

<table><thead><tr><th width="246.2421875">Table Name</th><th width="342.9296875">Description</th><th>PII Included</th></tr></thead><tbody><tr><td>deployment</td><td>History of location details for device and sensors: includes latitude, longitude, bearing.</td><td>No</td></tr><tr><td>events</td><td>Object Detection confirmed events. Generated by Frigate with required information attached. </td><td>Potential (if included in Frigate sub-label or attribute)</td></tr><tr><td>radar_dov</td><td>DetectedObjectVelocity (DOV) readings from Radar API, <code>ON</code> command</td><td>No</td></tr><tr><td>radar_timed_speed_counts</td><td>TimedSpeedCounts readings from Radar API <code>@O</code> command with `@</td><td>No</td></tr><tr><td>radar_raw_speed_magnitude</td><td>Show speeds and magnitudes in descending order by magnitude. <code>OS</code> and <code>OM</code> commands with <code>O3</code></td><td>No</td></tr><tr><td>radar_raw_speed_magnitude_single</td><td>Same as above, but only the first speed and magnitude value, for easier analysis. <code>OS</code> and <code>OM</code> commands with <code>O1</code></td><td>No</td></tr><tr><td>radar_oc_payload</td><td>Object detection capability, including vehicle length. <code>OC</code> command, with OPS9243 firmware</td><td>No</td></tr><tr><td>comments</td><td>Operator-entered time stamped entries for  Traffic Monitor status, deployment, location, conditions, construction, etc. Used for downstream analysis.</td><td>Potential (operator-entered free-text)</td></tr></tbody></table>

## How to access data

* Traffic Monitor Database:  On-device dashboard: [database](https://docs.trafficmonitor.ai/ui/database "mention") UI tab
* Frigate data: [Recording](https://docs.frigate.video/configuration/record) and [Snapshots](https://docs.frigate.video/configuration/record) follow [configuration](https://docs.trafficmonitor.ai/configuration/frigate-config) policies at `{{ tmsetup_codedir }}/docker/frigate/storage`
* Other apps, plugins, and sensors may also store data for their own operation