Agent integrates with DeepStack AI to provide intelligent object detection and Facial Recognition services. Agent uses DeepStack object detection to apply an intelligent filter to alerts to reduce the number of false alert events to near zero. You configure Agent as usual with a motion detector/ alert configuration and then add DeepStack alert filtering to ignore alerts that don't have specific objects in the frame - like car, person, dog etc.

To get started, install DeepStack for your platform (Docker). Use the GPU version if possible (you'll need an nvidia card for GPU acceleration). Make sure it's running in Vision Detection mode. To do that via the command line pass in "-e VISION-DETECTION=True".

Note: If you want to use facial recognition as well as object detection then start DeepStack with "-e VISION-DETECTION=True VISION-FACE=True".

When you have the deepstack server up and running and in object detection mode, switch back to Agent and go to Server Settings - Intelligence:

DeepStack

Put in the URL of the deepstack server here. Then edit a camera and go to the Alerts tab and click to configure the Intelligent Filter option:

DeepStack Configuration
  • Mask: Define an area to IGNORE (blue) - Use the tools at top right to create a mask to ignore. Important: The blue area is ignored by deepstack.
    This mask is used by both the alert filter and the object recognition system
  • Find: Enter a comma separated list of objects to look for. This setting must exactly match one or more of the Available options - for example cars or people won't work - it needs to be car or person (it's not case sensitive). Example: car, dog, person
  • Confidence: Minimum confidence of result. Anything below this value will be discarded (0-100)
  • Snapshots: Check this to take a photo and overlay results. These photos will be saved to Photos in Agent DVR.
  • Alert Condition: Choose from More Than, Less Than or Equal to (applies to Alert Number)
  • Alert Number: The number to use in the condition. For example, Find car More Than 0 will alert if any car is found. Find car Less Than 1 will alert if a car is not found.

Agent will generate motion events as normal (according to the motion detection tab) and when it does it will use DeepStack to look for relevant objects in the scene and decide if this is a real alert or not.

For Agent to apply the AI filter to new recordings, make sure Mode on the Recording tab when you edit the camera is set to Alert and not Detect.

Note: The first request into DeepStack can take a while to complete whilst it is initializing. You may see an error message about it not responding. Usually the second request will complete OK.

Configure the options on the screen. In this example we're blocking out the office using the Mask option but looking for people in the corridor. If a person is detected an alert will be generated.

When you are done click OK and then toggle the Enabled switch to start using intelligent filtering.


General Object Recognition

In addition to the intelligent alert filter Agent also offers general object recognition. This will generate events like "DeepStack Object Found" and "DeepStack Object Not Found" that you can perform actions on. General object recognition can be triggered by motion detection, alerts or via API

After configuring DeepStack, edit a camera and go to the Object Recognition tab:

Configure Object Recognition

  • Enabled: Switch Object Recognition on or off. You can also do this via the Schedule
  • Mode: When to process frames. You can choose from Alert, Motion which can be based on simple motion or a large object crossing a virtual trip wire. See Detectors, or None which is useful if you are triggering it via API.
  • Minimum Interval: How often to process frames when triggered.
  • Confidence: Minimum confidence of result. Anything below this value will be discarded (0-100)
  • Photos: Check this to take a photo and overlay results. These photos will be saved to Photos in Agent DVR.
  • Find: Enter the objects to look for in the scene (from the list) - example: car,dog. Make sure the labels match exactly.

Adding Object Recognition Actions

When Object Recognition runs Agent will generate events for "DeepStack Object Found" and "DeepStack Object Not Found" under Actions.

You can use these events to trigger actions such as calling a URL with the object labels or running an executable or publishing a message to an MQTT server. You can use the tags {AI} for the labels (if more than one they will be in a CSV list) or {AIJSON} for the full response JSON object from DeepStack.

DeepStack Troubleshooting

If DeepStack isn't filtering your recordings:

Check your Find setting exactly matches one or more of the Available options - for example cars or people won't work - it needs to be car or person (it's not case sensitive).

  • If you are getting errors (code 500 server errors) from DeepStack on Windows you may need to install https://aka.ms/vs/16/release/vc_redist.x64.exe
  • Check the master alert switch at the top left of Agent - It should show a closed padlock meaning Alerts are active. Click it to toggle.
  • Check that the recording mode on the recording tab when you edit the camera is set to Alert not Detect. DeepStack filtering only filters alert events.
  • Check that DeepStack filtering is enabled on the Alerts tab.
  • Check that DeepStack is running in DETECTION mode (VISION-DETECTION=True on the command line).
  • Try adjusting the Confidence level down in Agent as results may be being filtered
  • Check the logs at /logs.html for error messages from DeepStack. Possibly the DeepStack service is running on a different port or a firewall or antivirus is blocking access to it. If Agent can't contact DeepStack it will record the event.
  • Check the alert condition. If, for example you have it set to Alert when objects detected is less than 2 then it will alert when nothing is detected.
  • Check that DeepStack isn't overloading your computer. If DeepStack can't process the images quickly then it will time-out and Agent will give up waiting for a result and start recording. If DeepStack is timing out you will see errors in the logs. You may need to move the DeepStack service to another computer with more resources or a compatible GPU.