Adaptive Filtering of Visual Content in Distributed Publish/Subscribe Systems

Classic event matching techniques in large-scale Content-based Publish/Subscribe Systems mostly rely on predicate indexing or tree-based mechanisms for fast subscription evaluation. In the context of visual analytics, such techniques are limited in supporting subscriptions requiring expensive filtering operators over unstructured event types (i.e. images and videos). In this work, user subscriptions over visual content are answered as conjunctions of commutative Boolean filters where each filter is associated with a single high-level semantic concept that may be shared across multiple subscriptions. The shared-filter ordering problem has been previously studied in centralized data stream management systems; prior works propose approximation algorithms that achieve near-optimal cost reductions in the evaluation of overlapping queries. However, in a distributed publish/subscribe setting, even an optimal ordering of filter evaluations at brokers with high workloads can create bottlenecks and waste downstream resources. We present a distributed greedy algorithm that leverages existing routing methodologies to order and distribute the execution of filters across brokers on various dissemination paths. Experiments with several pub/sub workloads show 50% to 70% decrease in event latencies and noticeable improvements in resource utilization across the overlay.