AI Servers
To setup AI Servers click on the icon at the top left of the main Agent DVR UI and then click on Settings under Configuration and select AI Servers in the drop down menu and click Configure.
Auto Lock Tags on this screen will automatically lock (prevent storage management or accidental deletion) recordings with specified tags. To delete these recordings you will need to Unlock them (in the UI).
Agent DVR integrates with CodeProject.AI to provide various AI features including object recognition, face recognition, ALPR and super resolution (enhance). We also support PlateRecognizer.com as an ALPR provider. CodeProject.AI is open source and free and runs on pretty much every platform.
To get started, install an AI server for your platform and connect Agent DVR to it by clicking the Configure button and then Add.
You can add as many AI servers to Agent as you like. Cameras in Agent can be configured to use different AI servers for each function or you can just use one AI server for everything.
Configuring your Server
- Name: Give your server a name, eg: Cat Catcher
- AI Server URL: Enter the URL of your AI server. For example: http://localhost:5000
- API Key: Enter your key if you have set one up (optional)
- Timeout: Timeout in seconds for the requests to the server.
- Retry delay: Time in seconds before retrying requests to this server if they fail.
Click OK
Using PlateRecognizer
Configure LPR (ANPR or License Plate Recognition) in Agent DVR by going to Server Settings - AI Settings and enter the settings under Plate Recognizer. Sign up for a Free Trial on Plate Recognizer. No credit card required.
- URL: This is the URL to the service. The default is "https://api.platerecognizer.com/v1/plate-reader/" but you can enter your own server if you have your own instance of it running.
- Token: After creating a Plate Recognizer account, go to the Plate Recognizer Account Page and copy the API Token value.
- Regions: Leave this blank for default or enter a CSV list of regions
- Config: You can enter an extra config value here from the docs
Modules
In the AI server list (see above) you can access features to configure, test, edit and remove AI servers. Click on the configure button to list the modules that are available or installed on the selected server.
Agent DVR gets the current module list from your server and provides a UI to install, uninstall, configure and test each module. We've added support for all default CodeProject.UI modules although Agent DVR only uses a subset of them.
To use ALPR, Super Resolution or Face Recognition in Agent DVR you will need to click to install the relevant module from this page. Typically you won't need to worry about changing any settings on the modules but if you want to just click on the icon under each module.
Custom Models
CodeProject.AI comes with several in-built object detection models. To add your own models into CodeProject.AI just copy the model file (usually ends in .pt) to (on Windows) C:\Program Files\CodeProject\AI\AnalysisLayer\ObjectDetectionYolo\custom-models.
When it's there you should be able to access it by editing a camera, click on the Object Recognition tab under AI in the menu and then click on Get Models. If you have added your own model file you will need to manually add the object list to Object Classes as currently there is no way to read the available classes from the model file.
If your models are stored in a different folder you can change where CodeProject.AI is looking for them by editing the installed Object Recognition module settings.
Alerts
To alert on an AI detection event (from object recognition, LPR or Face recognition) you need to edit a camera and set the Mode on the Alerts tab to AI Detection. Then edit the AI configuration you want to use in the edit camera menu and click on the Alert Conditions button.
- Alert Pass-through: If your AI module stops working then this option will assume that motion events are alerts.
- Alert Zones: Agent compares the location of detected objects to zones you draw on the motion detector. Choose the zones you want to generate alerts for here. For example you could have zone 1 being the footpath and zone 2 being your driveway and you only want alerts about objects in your driveway.
- Prevent Repeat: Checking this will ignore repeat alerts until the object leaves and re-enters the zone
- Alert Condition: Select a condition. The default is "More Than". You can use Equals, More Than or Less Than
- Alert Number: Set a number. For example More than 0 will alert as soon as it finds something.
You then need to setup a trigger to trigger the AI to process frames from your camera. To do that choose a Mode on the AI configuration screen. Choose from Motion Detected, Alert, Interval or None. If you choose Motion Detected you will need to setup and enable the Motion Detector for the camera to generate motion events. Alternatively you can choose Interval and have Agent just send a frame off to the AI server every few seconds (This can use a lot of CPU though).
Alert Filtering
Object Recognition
Object Recognition uses an AI server (we recommend CodeProject.AI) to recognise specific objects in your video feeds and generates events, raises alerts or acts as a filter on motion alerts.
- Enabled: Enable or disable the AI process
- AI Server: Choose from one of your configured servers or use the Default (first).
- Mode: Choose what will trigger the AI process to run
- Alert Conditions: See Alerts
- Use Snapshot URI: Check this to get a high res frame from your camera instead of using the current live stream frame
- Overlay: This draws the AI results on the live video stream
- Minimum Interval: The minimum time in seconds between requests to the server (use this to reduce load)
- Confidence: The minimum percentage confidence to qualify a result as a recognised object (0 - 100)
Model
- Discover: Click to get installed models from your server (CodeProject.AI only)
- Model Endpoint: Choose from one of the models or use the default at /v1/vision/detection
- Object Classes: This will automatically populate with relevant object classes or you can enter your own
- Find: Click this button to specify which objects specifically you want to look for
Photos
see photos
AI Photos
AI processes can take photos when objects are recognised and save them, crop them, FTP them etc.
To set this up see the Photos option at the bottom of each AI configuration tab when editing a camera. Enable Photos and click to configure it.
- Label: Agent will overlay boxes on the image and label what it found.
- Crop: Agent crops the image to the detected areas and saves multiple images with each detected region.
- FTP: Uploads the saved images to the camera's configured FTP server.
- Prevent Repeat: Agent won't save multiple copies of the same object until it has left the motion zone
LPR (ALPR)
LPR (ALPR/ ANPR) uses an AI server (we recommend CodeProject.AI and PlateRecognizer.com) to recognise and read license plates from cars in your video feeds and generates events, raises alerts or acts as a filter on motion alerts.
- Enabled: Enable or disable the AI process
- AI Server: Choose from one of your configured servers or use the Default (first). Agent supports LPR via CodeProject.AI or PlateRecogniser.com. See Server Settings to configure servers.
- Mode: Choose what will trigger the AI process to run
- Alert Conditions: See Alerts
- Use Snapshot URI: Check this to get a high res frame from your camera instead of using the current live stream frame
- Overlay: This draws the AI results on the live video stream
- Minimum Interval: The minimum time in seconds between requests to the server (use this to reduce load)
- Confidence: The minimum percentage confidence to qualify a result as a recognised object (0 - 100)
- License Plates: A comma separated list of plates to find or a URL to a CSV text file containing plates (optional). If plates are provided Agent will generate License Plate Recognized and License Plate Not Recognized events that you can add actions to in the actions tab when editing the camera.
- Reload Interval: If a URL is provided, how often to reload it.
- Normalise: Replaces commonly misidentified plates to improve matching.
- Make, Model and Color: Check this only if you have a paid plan that supports it on PlateRecognizer.com. It's not included in the free plan. Make model and color will be requested with the image and returned in {AIJSON} in Agent Actions.
Photos
see photos
Face Recognition
Face Recognition uses an AI server (we recommend CodeProject.AI) to recognise specific faces in your video feeds and generates events, raises alerts or acts as a filter on motion alerts. You can add/ edit and delete faces using your camera or by uploading images. See Edit Faces in this tab.
- Enabled: Enable or disable the AI process
- AI Server: Choose from one of your configured servers or use the Default (first).
- Mode: Choose what will trigger the AI process to run
- Alert Conditions: See Alerts
- Use Snapshot URI: Check this to get a high res frame from your camera instead of using the current live stream frame
- Overlay: This draws the AI results on the live video stream
- Minimum Interval: The minimum time in seconds between requests to the server (use this to reduce load)
- Confidence: The minimum percentage confidence to qualify a result as a recognised object (0 - 100)
- Edit Faces: Click this to upload images to the server database so it can recognise people. You can upload images from disk or take photo's with the selected camera.
Photos
see photos
Changing the AI language
To change the language that Agent uses when displaying labels and tagging recordings see Server Menu - Settings - Playback - Default Language.
Adding Actions
When an AI process runs Agent will generate events that you can use to trigger Actions. For example, Object Recognition will generate events for "Object Found" and "Object Not Found". Each AI system in Agent generate it's own events.
You can use these events to trigger actions such as calling a URL with the object labels or running an executable or publishing a message to an MQTT server. You can use the tags {AI} for the labels (if more than one they will be in a CSV list) or {AIJSON} for the full response JSON object from CodeProject.AI.
AI Server Troubleshooting
If AI isn't filtering your recordings:
Check your Find setting exactly matches one or more of the Available options.
- Check the master alert switch at the top left of Agent - It should show a closed padlock meaning Alerts are active. Click it to toggle.
- Check that the recording mode on the recording tab when you edit the camera is set to Alert not Detect.
- Check that the alert mode on the Alerts tab in the camera is set to AI Detection
- Try adjusting the Confidence level down under Object Recognition as results may be being filtered
- Check the logs at /logs.html for error messages from the server. Possibly the AI service is running on a different port or a firewall or antivirus is blocking access to it.
- Check the alert condition. If, for example you have it set to Alert when objects detected is less than 2 then it will alert when nothing is detected.
- Check that the AI server isn't overloading your computer. If the AI server can't process the images quickly then it will time-out and Agent DVR will give up waiting for a result. If the AI server is timing out you will see errors in the logs. You may need to move the AI service to another computer with more resources or a compatible GPU.
- If AI is detecting every class of object this usually means it's not working with your GPU. Check your GPU drivers or try switching to the CPU version of the AI module.