CityBench: A configurable benchmark to evaluate RSP engines using smart city datasets

Muhammad Intizar Ali, Feng Gao, Alessandra Mileo

Research output: Chapter in Book or Conference Publication/ProceedingConference Publicationpeer-review

120 Citations (Scopus)

Abstract

With the growing popularity of Internet of Things (IoT) and IoT-enabled smart city applications, RDF stream processing (RSP) is gaining increasing attention in the Semantic Web community. As a result, several RSP engines have emerged, which are capable of processing semantically annotated data streams on the fly. Performance, correctness and technical soundness of few existing RSP engines have been evaluated in controlled settings using existing benchmarks like LSBench and SRBench. However, these benchmarks focus merely on features of the RSP query languages and engines, and do not consider dynamic application requirements and data-dependent properties such as changes in streaming rate during query execution or changes in application requirements over a period of time. This hinders wide adoption of RSP engines for real-time applications where data properties and application requirements play a key role and need to be characterised in their dynamic setting, such as in the smart city domain. In this paper, we present CityBench, a comprehensive benchmarking suite to evaluate RSP engines within smart city applications and with smart city data. CityBench includes real-time IoT data streams generated from various sensors deployed within the city of Aarhus, Denmark. We provide a configurable testing infrastructure and a set of continuous queries covering a variety of data- and application- dependent characteristics and performance metrics, to be executed over RSP engines using CityBench datasets. We evaluate two state of the art RSP engines using our testbed and discuss our experimental results. This work can be used as a baseline to identify capabilities and limitations of existing RSP engines for smart city applications.

Original languageEnglish
Title of host publicationThe Semantic Web – ISWC 2015 - 14th International Semantic Web Conference, Proceedings
EditorsMarcelo Arenas, Oscar Corcho, Elena Simperl, Markus Strohmaier, Mathieu d’Aquin, Kavitha Srinivas, Paul Groth, Michel Dumontier, Jeff Heflin, Krishnaprasad Thirunarayan, Steffen Staab
PublisherSpringer-Verlag
Pages374-389
Number of pages16
ISBN (Print)9783319250090
DOIs
Publication statusPublished - 2015
Externally publishedYes
Event14th International Semantic Web Conference, ISWC 2015 - Bethlehem, United States
Duration: 11 Oct 201515 Oct 2015

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9367
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference14th International Semantic Web Conference, ISWC 2015
Country/TerritoryUnited States
CityBethlehem
Period11/10/1515/10/15

Fingerprint

Dive into the research topics of 'CityBench: A configurable benchmark to evaluate RSP engines using smart city datasets'. Together they form a unique fingerprint.

Cite this