This guide explains why false alarms are generated by Avlytics GO! product and how to configure the device to mitigate these false alarms.
Let us first understand how the software on the device operates to better understand why there are false alarms.
The Avlytics GO! product is a pre trained object classification device.
The device is supplied with a basic understanding of a few objects, these include Human, Vehicle, Animal and Background.
The training in the pretrained model has hundreds of sample images for each category.
The sample images have what we call overlapping features, these features include paving , grass, walls, fences, sky etc.
Why is this relevant ?
When the Training model detects movement and makes a classification, the model will predict a confidence score of a particular classification in each category.
This means each event has a Match Score for each category.
Example.
When the device detects a Human walking in the camera's field of view, The training model will examine all the features in the image and make a prediction on what the object is most likely to be.
In this example case the model will predict that the Image has a
Match Score of 45.0 for Background
Match Score of 23.76 for Animal
Match Score of 32.98 for Vehicle
Match Score of 99.7 for Human
This is to be expected as each of the objects detected could all have similar background composition which will attribute to the features that the Model is using to make a prediction on what the object is.
The model selects the classification with the highest match score as the winner and an alert notification will be sent to the notification channels with the classification being the category with the highest score.
In this case.
Human presence ( Match score : 99.7 )
If for example a plastic packet was blowing in the wind and the device makes a prediction, the most likely outcome is that the device will make a prediction with the following scores:
Match Score of 45.0 for Background
Match Score of 43.76 for Animal
Match Score of 23.98 for Vehicle
Match Score of 54.7 for Human
The user will notice that the scores are all very close in value.
This means the device is detecting features that are common in all categories but none of the categories
have a very distinct feature that identifies this object as that category.
In this case the device is unsure of what the object is but is programmed to send an Alert Notification for the Highest Score. The Device sends a Human presence (Match: 54.7) notification which appears to be a false alarm. This is in fact not a false alarm as it is a notification that this object could be a Human as it has more features in common with a Human than any other category but the Model is not very confident that it is actually Human. This is a great tool that allows the software to detect obscured Humans or Humans that try to disguise themselves, but does generate a higher "false" alarm rate.
To mitigate these false or low confidence alarms, the user is able to set a threshold for the match score in the device's rules configuration.
This instructs the device to only send notifications that have a High Confidence score and to suppress or not send any notification on an object where the confidence is less than a predefined value.
We recommend performing a walk test to easily identify what the limits of detection are by examining the Match scores associated with each human detected in the scene during the walk test.
This can be done by logging in to SeeingAME Hub : http://192.168.7.1:8300 and opening the training page. Filter the Training page to the device and channel where the walk test was done.
Examine the alerts to determine what the lowest match score was for a human in the area that a human is to be detected in.
In the above example (when expanding the Info) by clicking on the Info link we see the match score of each category for every detection. From these Scores we are able to identify what the limit should be for the device to filter our perceived false alarms.
To filter out these alarms, open the Client page and Click on the device that you would like to configure.
From this menu the user is able to configure all channel, rules and region configuration.
Click on the device to be configured.
Once a device has been selected the following screen will be displayed.
Select the RULES option to open the device’s rules configuration.
To edit a Device’s rules , Click on the blue edit icon to edit a specific rule.
In the Rules configuration the user will notice that the channel the rule relates to is selected, the region of interest is selected, the list of classifications that should be sent as notifications, the Date and Time arming schedule and a Probability option.
The probability is where you set the threshold for the match score.
Select the greater than symbol ( > ) to specify, that an alert with a match score greater than .(x Value).
should be sent as a notification.
Set the value in the input box to the right.
This will ensure that only alerts with a confidence or match score greater than the value that has been set are sent as notifications.