Development and Optimization <span>of Edge Computer Vision Algorithms</span> | SQUAD Tech

Development and Optimization of Edge Computer Vision Algorithms

SQUAD replaced a third party motion detection library with on-device deep learning models, reduced licensing and cloud costs, and increased the quality of motion detection in smart security cameras.

20+ edge CV projects delivered

completed since 2016

Real-time multi-class motion detection

running fully on-device

Improved motion detection rating

from three stars to above four stars

Client at a Glance

Service Type

Edge computer vision development and optimization

Industry

Consumer electronics and smart security cameras

Engagement

Long term collaboration

Region

Global

The client is a global consumer electronics brand that produces smart indoor and outdoor security cameras for residential and commercial use.

Challenge

The client used a third party library with a motion detection algorithm in all camera products.

This approach created several issues:

Licensing was applied per camera, which resulted in significant ongoing cost.

The algorithm relied on cascading filters and did not use the available computational resources effectively.

Accuracy and processing problems were observed and remained unresolved.

To address these points and avoid redundant on device computation, the client introduced a cloud based motion detection pipeline. This reduced dependence on the original library, but introduced new cloud costs and latency for motion based alerts.

The client needed a way to:

Remove repeated licensing payments.

Decrease reliance on cloud processing for motion detection.

Use device resources in a more efficient way while keeping latency low.

SQUAD proposed and implemented a new approach based on deep neural networks for motion detection that run directly on the camera at the edge.

Solution

Since 2016, SQUAD has completed more than twenty projects that deploy computer vision models to the edge for this client. The process can be described in several stages.

Model and architecture selection

Reviewed available model families that fit the hardware constraints of each camera platform.

Selected an architecture that combined classifier and detector models in line with the motion detection requirements.

Hardware and platform support

Studied and integrated software development kits and neural accelerators for each System on Chip family, including Ambarella, Qualcomm, OmniVision and SigmaStar.

Training and fine tuning

Adapted final layers to detect a predefined list of object classes such as person, vehicle, animal and package.

Repeated training and validation cycles to reach the required Precision and Recall values on validation and test datasets.

Performance optimization

Applied the following optimization steps to reach real time performance on the camera:

Pruned the network to remove layers and connections that were not needed.

Applied quantization to lower compute and memory usage.

Used platform specific compilers and accelerators for each System on Chip.

Verified that the deployed model on the device met the agreed performance and accuracy targets.

Quality assurance and end-to-end testing

Engaged the AI quality assurance team throughout the project:

Tested each release candidate model.

Performed end-to-end testing on physical devices that ran on device detection in realistic environments.

Technologies and frameworks

The work relied on the following components:

System on Chip families: Ambarella, Qualcomm, OmniVision and SigmaStar

Deep neural networks for object detection in video streams

Python, TensorFlow and PyTorch for training, pruning and quantization

In house tools for dataset management, model fine tuning and evaluation

SQUAD Labs test area for motion detection experiments, including an automated mannequin for repeatable motion scenarios

Results & Impact

technical outcomes

Edge CV delivered at scale

More than twenty edge computer vision projects have been completed for this client since 2016, covering multiple camera models and hardware platforms.

Real-time on-device detection

Motion events are processed in real time on the camera with multi-class object detection, so standard scenarios no longer require cloud inference.

business outcomes

Lower licensing costs

Recurring payments for the third-party motion detection library were removed, reducing the overall cost of ownership for the camera portfolio.

Reduced cloud spend

Cloud infrastructure costs for motion detection decreased because fewer events need to be processed in the cloud and more work is done directly on the device.

customer outcomes

More reliable notifications

The number of false alerts decreased, and push notifications for motion events arrive faster, since computation is carried out on the device.

Higher satisfaction with motion detection

Customer rating for motion detection increased from about three stars to above four stars, supported by a motion sensitivity setting that lets users adjust the system to their environment.