Real-Time Context-Aware Early Filtering for High-Definition Video Analytics on Commodity Edge Devices Using GenAI for Data Augmentation

This work proposes a fast and accurate early filtering pipeline for video analytics in commodity Edge devices for Smart-Cities applications. This pipeline can run in real-time even on a small and GPU-less device such as a Raspberry Pi, while maintaining a good accuracy for video analytics queries. In addition to a novel Edge optimized pre-processing method, the pipeline uses a context-aware binary model, which is fine-tuned using semi-automatic synthetic data augmentation, Generative AI, and Cut-and-Paste techniques to contextualize the model to the input camera background and the Objects of Interest (e.g., car or person) from a user's video analytics query, in a fast process that requires only 10 seconds of original footage for training. This makes it the first Edge filtering with specialized models with a viable online training solution. Compared to a baseline state-of-art Nano-YoloV5 model, the proposed early filtering pipeline in its high speed profile shows an 48.8x increase in speed and is the first of its kind that is able to run on physical hardware (i.e., non-simulated) commodity Edge devices at more than 80 FPS in HD (1920 × 1080) resolution, with a small accuracy loss of 5% compared to the baseline. On the high accuracy setting the pipeline still runs at more than 41 FPS (26.9x faster than Nano-Yolo) and shows an increase of 2.5% in accuracy.