AI Servers

To setup AI Servers click on the icon at the top left of the main Agent DVR UI and then click on Settings under Configuration and select AI Servers in the drop down menu and click Configure.

Auto Lock Tags on this screen will automatically lock (prevent storage management or accidental deletion) recordings with specified tags. To delete these recordings you will need to Unlock them (in the UI).

Agent DVR integrates with CodeProject.AI to provide various AI features including object recognition, face recognition, ALPR and super resolution (enhance). We also support PlateRecognizer.com as an ALPR provider. CodeProject.AI is open source and free and runs on pretty much every platform.

To get started, install an AI server for your platform and connect Agent DVR to it by clicking the Configure button and then Add.

You can add as many AI servers to Agent as you like. Cameras in Agent can be configured to use different AI servers for each function or you can just use one AI server for everything.

Warning: AI processing can use a lot of CPU. Make sure the computer you install it on has enough power to run it.
Configuring your Server
  • Name: Give your server a name, eg: Cat Catcher
  • AI Server URL: Enter the URL of your AI server. For example: http://localhost:5000
  • API Key: Enter your key if you have set one up (optional)
  • Timeout: Timeout in seconds for the requests to the server.
  • Retry delay: Time in seconds before retrying requests to this server if they fail.

Click OK

Using PlateRecognizer

Configure LPR (ANPR or License Plate Recognition) in Agent DVR by going to Server Settings - AI Settings and enter the settings under Plate Recognizer. Sign up for a Free Trial on Plate Recognizer. No credit card required.

  • URL: This is the URL to the service. The default is "https://api.platerecognizer.com/v1/plate-reader/" but you can enter your own server if you have your own instance of it running.
  • Token: After creating a Plate Recognizer account, go to the Plate Recognizer Account Page and copy the API Token value.
  • Regions: Leave this blank for default or enter a CSV list of regions
  • Config: You can enter an extra config value here from the docs

Modules

In the AI server list (see above) you can access features to configure, test, edit and remove AI servers. Click on the configure button to list the modules that are available or installed on the selected server.

Agent DVR gets the current module list from your server and provides a UI to install, uninstall, configure and test each module. We've added support for all default CodeProject.UI modules although Agent DVR only uses a subset of them.

Only install one of the Object Recognition module options. Read the description for each one to decide which is best for your system.

To use ALPR, Super Resolution or Face Recognition in Agent DVR you will need to click to install the relevant module from this page. Typically you won't need to worry about changing any settings on the modules but if you want to just click on the icon under each module.

AI Alert Filtering

To setup Alert filtering you will need to setup an AI server, and follow these steps:

  • Set up and enable a Motion Detector. For minimal CPU usage use the Simple detector. Make sure at least one zone is defined covering the area you want to monitor.
  • Set the Mode on the Alerts tab to Actions Only. Enable Alerts.
  • Enable Object Recognition (you can use any AI detector to filter alerts) on the Object Recognition tab. Set the mode to Motion Detected, choose a model to use and then click the Find button to choose the objects you want to look for. For example, Person, Dog, Car etc
  • Next select Actions in the tab menu and add an Action for the event AI: Object Found.

    If you have setup motion zones here you can specify which zone you want to look in - for example you could have a zone covering your driveway and a different zone on the road. Choosing the driveway zone here would cause Agent to only alert if a car is detected on your driveway.

    Under Tasks click the Add button to add a task for Alert. Click OK twice to confirm.

Agent will then run AI object recognition when the motion detector sees movement. If it detects an object you specified in a zone you specified it will then run an action to trigger an alert. If you don't select any zones then it will alert on any zone.

You can setup alert filters in a similar way for LPR Recognition, Face Recognition or Audio Recognition

You can use AI object recognition without a motion detection trigger - just set the Mode on Object Recognition to Interval. Be sure to check the resource usage that AI recognition puts on your hardware and adjust accordingly.

You can setup multiple actions for different objects in different zones. Use the {AI} tag in actions to access the name of the object it has recognized.

If your AI server isn't responding to requests and you are using it to filter motion detection Alerts then Agent will assume it is a valid alert event until the AI server comes back online. You can toggle this behavior with the Motion Pass-through switch under Object Recognition.
Using filters in your actions you can perform different actions based on what Agent detects in what zones. For example you could have it say "intruder detected" if it sees a person in a doorway or play the sound of a dog barking if it sees a cat on your couch.

Object Recognition

Object Recognition uses an AI server (we recommend CodeProject.AI) to recognise specific objects in your video feeds and generates events, raises alerts or acts as a filter on motion alerts.

  • Enabled: Enable or disable the AI process
  • AI Server: Choose from one of your configured servers or use the Default (first).
  • Mode: Choose what will trigger the AI process to run
  • Motion Pass-through: If you are using an AI server to filter alerts but your AI server is down then pass the alert event on (don't filter it). Note that if your alert actions are based on zones or detected objects then they won't fire without your AI server working.
  • Use Snapshot URI: Check this to get a high res frame from your camera instead of using the current live stream frame
  • Resize Mode: By default Agent sends in the full frame to the AI server. If this is high resolution the server might take a while to respond and you may get timeouts. Agent can resize the image before it sends it in to reduce the load on the AI server. For example, using Small here can result in much faster results from AI processing but potentially less accurate.
  • Overlay: This draws the AI results on the live video stream
  • Minimum Interval: The minimum time in seconds between requests to the server (use this to reduce load)
  • Confidence: The minimum percentage confidence to qualify a result as a recognised object (0 - 100)
  • Check Corners: See Checking Corners
Models
  • Discover: Click to get installed models from your server (CodeProject.AI only)
  • Model Endpoint: Choose from one of the models or use the default at /v1/vision/detection
  • Object Classes: This will automatically populate with relevant object classes or you can enter your own
  • Find: Click this button to specify which objects specifically you want to look for
  • Ignore Static Objects: This option will ignore objects found repeatedly in the same location. For example use this if you want to be alerted when a car parks in your driveway but not alerted again until it leaves.
Custom Models

CodeProject.AI comes with several in-built object detection models. To add your own models into CodeProject.AI just copy the model file (usually ends in .pt) to (on Windows) C:\Program Files\CodeProject\AI\AnalysisLayer\ObjectDetectionYolo\custom-models.

When it's there you should be able to access it by clicking the Discover button. You will need to manually add the object list to Object Classes as currently there is no way to read the available classes from the model file.

If your models are stored in a different folder you can change where CodeProject.AI is looking for them by editing the installed Object Recognition module settings.

Actions

Object Recognition generates AI: Object Found and AI: Object Not Found events that you can use in Actions.

Photos

see photos

AI Photos

AI processes can take photos when objects are recognised and save them, crop them, FTP them etc.

To set this up see the Photos option at the bottom of each AI configuration tab when editing a camera. Enable Photos and click to configure it.

  • Label: Agent will overlay boxes on the image and label what it found.
  • Crop: Agent crops the image to the detected areas and saves multiple images with each detected region.
  • FTP: Uploads the saved images to the camera's configured FTP server.
  • Prevent Repeat: Agent won't save multiple copies of the same object until it has left the motion zone
Be careful with AI photos as the wrong settings can result in millions of images being saved. Check how often it is saving images after you set it up.

Checking Corners

Agent will check the center point of the detected object against your zone configuration to work out if it should process alerts/ actions. It can also check the corners of the bounding box of the object. Using this feature you can set a percentage of the distance from the center point to the corner of the bounding box to check the zone. Basically 0 = center point only, 100 = check all corners and 50 = check the points halfway to each corner of the bounding rectangle. If you are getting lots of event notifications where the object doesn't look like it's in the zone then set Check Corners 0.

LPR (ALPR)

LPR (ALPR/ ANPR) uses an AI server (we recommend CodeProject.AI and PlateRecognizer.com) to recognise and read license plates from cars in your video feeds and generates events, raises alerts or acts as a filter on motion alerts.

  • Enabled: Enable or disable the AI process
  • AI Server: Choose from one of your configured servers or use the Default (first). Agent supports LPR via CodeProject.AI or PlateRecogniser.com. See Server Settings to configure servers.
  • Mode: Choose what will trigger the AI process to run
  • Use Snapshot URI: Check this to get a high res frame from your camera instead of using the current live stream frame
  • Overlay: This draws the AI results on the live video stream
  • Minimum Interval: The minimum time in seconds between requests to the server (use this to reduce load)
  • Confidence: The minimum percentage confidence to qualify a result as a recognised object (0 - 100)
  • Check Corners: See Checking Corners
  • License Plates: A comma separated list of plates to find or a URL to a CSV text file containing plates (optional). If plates are provided Agent will generate License Plate Recognized and License Plate Not Recognized events that you can add actions to in the actions tab when editing the camera.
  • Reload Interval: If a URL is provided, how often to reload it.
  • Normalise: Replaces commonly misidentified plates to improve matching.
  • Make, Model and Color: Check this only if you have a paid plan that supports it on PlateRecognizer.com. It's not included in the free plan. Make model and color will be requested with the image and returned in {AIJSON} in Agent Actions.
Actions

LPR generates AI: License Plate Recognised and AI: License Plate Not Recognised events that you can use in Actions.

Photos

see photos

Face Recognition

Face Recognition uses an AI server (we recommend CodeProject.AI) to recognise specific faces in your video feeds and generates events, raises alerts or acts as a filter on motion alerts. You can add/ edit and delete faces using your camera or by uploading images. See Edit Faces in this tab.

  • Enabled: Enable or disable the AI process
  • AI Server: Choose from one of your configured servers or use the Default (first).
  • Mode: Choose what will trigger the AI process to run
  • Use Snapshot URI: Check this to get a high res frame from your camera instead of using the current live stream frame
  • Overlay: This draws the AI results on the live video stream
  • Minimum Interval: The minimum time in seconds between requests to the server (use this to reduce load)
  • Confidence: The minimum percentage confidence to qualify a result as a recognised object (0 - 100)
  • Check Corners: See Checking Corners
  • Edit Faces: Click this to upload images to the server database so it can recognise people. You can upload images from disk or take photo's with the currently selected camera. When adding faces ensure that only one face is visible and is clearly defined in the image.
Actions

Face Recognition generates AI: Face Recognised and AI: Face Not Recognised events that you can use in Actions.

Photos

see photos

Audio Recognition

You can use AI based audio recognition to respond to recognized sounds from your microphone or audio streams in Agent DVR. To set this up you will need to download a model file from Server Settings - Data - AI Audio Model (requires an iSpyConnect.com account)

  • Enabled: Enable or disable the AI process
  • Mode: Choose what will trigger the AI process to run
  • Confidence: The minimum percentage confidence to qualify a result as a recognised object (0 - 100)
  • Overlay: This displays the AI results on the live audio visualization
  • Listen For: Choose the events to listen out for.

Clicking on Listen For displays the available sounds Agent can recognize. Click to select the sounds you want to detect.

You can use the Action AI: Sound Recognized to perform tasks when a sound is recognized.

You can also use audio recognition to filter alerts in the same way as for cameras.

Changing the AI language

To change the language that Agent uses when displaying labels and tagging recordings see Server Menu - Settings - Playback - Default Language.

Adding Actions

When an AI process runs Agent will generate events that you can use to trigger Actions. For example, Object Recognition will generate events for "Object Found" and "Object Not Found". Each AI system in Agent generate it's own events.

You can use these events to trigger actions such as raising an Alert on the device, calling a URL with the object labels or running an executable or publishing a message to an MQTT server. You can use the tags {AI} for the labels (if more than one they will be in a CSV list) or {AIJSON} for the full response JSON object from CodeProject.AI.

AI Server Troubleshooting

If AI isn't filtering your recordings:

Check your Find setting exactly matches one or more of the Available options.

  • Check the master alert switch at the top left of Agent - It should show a closed padlock meaning Alerts are active. Click it to toggle.
  • Check that the recording mode on the recording tab when you edit the camera is set to Alert not Detect.
  • Check that the alert mode on the Alerts tab in the camera is set to Actions Only
  • Try adjusting the Confidence level down under Object Recognition as results may be being filtered
  • Check the logs at /logs.html for error messages from the server. Possibly the AI service is running on a different port or a firewall or antivirus is blocking access to it.
  • Check that the AI server isn't overloading your computer. If the AI server can't process the images quickly then it will time-out and Agent DVR will give up waiting for a result. If the AI server is timing out you will see errors in the logs. You may need to move the AI service to another computer with more resources or a compatible GPU.
  • If AI is detecting every class of object this usually means it's not working with your GPU. Check your GPU drivers or try switching to the CPU version of the AI module.