Representing interoperable provenance descriptions for ETL workflows

André Freitas; Benedikt Kömpgen; João Gabriel Oliveira; Seán O’Riain; Edward Curry

doi:10.1007/978-3-662-46641-4_4

Representing interoperable provenance descriptions for ETL workflows

André Freitas
, Benedikt Kömpgen
, João Gabriel Oliveira
, Seán O’Riain
, Edward Curry

Computer Science

University of Galway
Institute of Toxicology and Genetics

Research output: Chapter in Book or Conference Publication/Proceeding › Conference Publication › peer-review

Abstract

The increasing availability of data on the Web provided by the emergence of Web 2.0 applications and, more recently by Linked Data, brought additional complexity to data management tasks, where the number of available data sources and their associated heterogeneity drastically increases. In this scenario, where data is reused and repurposed on a new scale, the pattern expressed as Extract-Transform-Load (ETL) emerges as a fundamental and recurrent process for both producers and consumers of data on the Web. In addition to ETL, provenance, the representation of source artifacts, processes and agents behind data, becomes another cornerstone element forWeb data management, playing a fundamental role in data quality assessment, data semantics and facilitating the reproducibility of data transformation processes. This paper proposes the convergence of these two Web data management concerns, introducing a principled provenance model for ETL processes in the form of a vocabulary based on the Open Provenance Model (OPM) standard and focusing on the provision of an interoperable provenance model for ETL environments. The proposed ETL provenance model is instantiated in a real-world sustainability reporting scenario.

Original language	English
Title of host publication	The Semantic Web
Subtitle of host publication	ESWC 2012 Satellite Events - Revised Selected Papers
Editors	Alexandre Passant, Barry Norton, Emanuele Della Valle, Raphael Troncy, Irini Fundulaki, Elena Simperl, Dunja Mladenic
Publisher	Springer-Verlag
Pages	43-57
Number of pages	15
ISBN (Print)	9783662466407
DOIs	https://doi.org/10.1007/978-3-662-46641-4_4
Publication status	Published - 2015
Event	Extended Semantic Web Conference, ESWC 2012 - Heraklion, Greece Duration: 27 May 2012 → 31 May 2012

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	7540
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	Extended Semantic Web Conference, ESWC 2012
Country/Territory	Greece
City	Heraklion
Period	27/05/12 → 31/05/12

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 9 Industry, Innovation, and Infrastructure
SDG 12 Responsible Consumption and Production

Keywords

Data transformation
ETL
Linked Data · Web
Provenance

Access to Document

10.1007/978-3-662-46641-4_4

Cite this

Freitas, A., Kömpgen, B., Oliveira, J. G., O’Riain, S., & Curry, E. (2015). Representing interoperable provenance descriptions for ETL workflows. In A. Passant, B. Norton, E. D. Valle, R. Troncy, I. Fundulaki, E. Simperl, & D. Mladenic (Eds.), The Semantic Web: ESWC 2012 Satellite Events - Revised Selected Papers (pp. 43-57). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7540). Springer-Verlag. https://doi.org/10.1007/978-3-662-46641-4_4

Freitas, André ; Kömpgen, Benedikt ; Oliveira, João Gabriel et al. / Representing interoperable provenance descriptions for ETL workflows. The Semantic Web: ESWC 2012 Satellite Events - Revised Selected Papers. editor / Alexandre Passant ; Barry Norton ; Emanuele Della Valle ; Raphael Troncy ; Irini Fundulaki ; Elena Simperl ; Dunja Mladenic. Springer-Verlag, 2015. pp. 43-57 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{d5a269cea1dc46b5a0cf32aab8239961,

title = "Representing interoperable provenance descriptions for ETL workflows",

abstract = "The increasing availability of data on the Web provided by the emergence of Web 2.0 applications and, more recently by Linked Data, brought additional complexity to data management tasks, where the number of available data sources and their associated heterogeneity drastically increases. In this scenario, where data is reused and repurposed on a new scale, the pattern expressed as Extract-Transform-Load (ETL) emerges as a fundamental and recurrent process for both producers and consumers of data on the Web. In addition to ETL, provenance, the representation of source artifacts, processes and agents behind data, becomes another cornerstone element forWeb data management, playing a fundamental role in data quality assessment, data semantics and facilitating the reproducibility of data transformation processes. This paper proposes the convergence of these two Web data management concerns, introducing a principled provenance model for ETL processes in the form of a vocabulary based on the Open Provenance Model (OPM) standard and focusing on the provision of an interoperable provenance model for ETL environments. The proposed ETL provenance model is instantiated in a real-world sustainability reporting scenario.",

keywords = "Data transformation, ETL, Linked Data · Web, Provenance",

author = "Andr{\'e} Freitas and Benedikt K{\"o}mpgen and Oliveira, \{Jo{\~a}o Gabriel\} and Se{\'a}n O{\textquoteright}Riain and Edward Curry",

note = "Publisher Copyright: {\textcopyright} Springer-Verlag Berlin Heidelberg 2015.; Extended Semantic Web Conference, ESWC 2012 ; Conference date: 27-05-2012 Through 31-05-2012",

year = "2015",

doi = "10.1007/978-3-662-46641-4\_4",

language = "English",

isbn = "9783662466407",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer-Verlag",

pages = "43--57",

editor = "Alexandre Passant and Barry Norton and Valle, \{Emanuele Della\} and Raphael Troncy and Irini Fundulaki and Elena Simperl and Dunja Mladenic",

booktitle = "The Semantic Web",

}

Freitas, A, Kömpgen, B, Oliveira, JG, O’Riain, S & Curry, E 2015, Representing interoperable provenance descriptions for ETL workflows. in A Passant, B Norton, ED Valle, R Troncy, I Fundulaki, E Simperl & D Mladenic (eds), The Semantic Web: ESWC 2012 Satellite Events - Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 7540, Springer-Verlag, pp. 43-57, Extended Semantic Web Conference, ESWC 2012, Heraklion, Greece, 27/05/12. https://doi.org/10.1007/978-3-662-46641-4_4

Representing interoperable provenance descriptions for ETL workflows. / Freitas, André; Kömpgen, Benedikt; Oliveira, João Gabriel et al.
The Semantic Web: ESWC 2012 Satellite Events - Revised Selected Papers. ed. / Alexandre Passant; Barry Norton; Emanuele Della Valle; Raphael Troncy; Irini Fundulaki; Elena Simperl; Dunja Mladenic. Springer-Verlag, 2015. p. 43-57 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7540).

Research output: Chapter in Book or Conference Publication/Proceeding › Conference Publication › peer-review

TY - GEN

T1 - Representing interoperable provenance descriptions for ETL workflows

AU - Freitas, André

AU - Kömpgen, Benedikt

AU - Oliveira, João Gabriel

AU - O’Riain, Seán

AU - Curry, Edward

N1 - Publisher Copyright: © Springer-Verlag Berlin Heidelberg 2015.

PY - 2015

Y1 - 2015

N2 - The increasing availability of data on the Web provided by the emergence of Web 2.0 applications and, more recently by Linked Data, brought additional complexity to data management tasks, where the number of available data sources and their associated heterogeneity drastically increases. In this scenario, where data is reused and repurposed on a new scale, the pattern expressed as Extract-Transform-Load (ETL) emerges as a fundamental and recurrent process for both producers and consumers of data on the Web. In addition to ETL, provenance, the representation of source artifacts, processes and agents behind data, becomes another cornerstone element forWeb data management, playing a fundamental role in data quality assessment, data semantics and facilitating the reproducibility of data transformation processes. This paper proposes the convergence of these two Web data management concerns, introducing a principled provenance model for ETL processes in the form of a vocabulary based on the Open Provenance Model (OPM) standard and focusing on the provision of an interoperable provenance model for ETL environments. The proposed ETL provenance model is instantiated in a real-world sustainability reporting scenario.

AB - The increasing availability of data on the Web provided by the emergence of Web 2.0 applications and, more recently by Linked Data, brought additional complexity to data management tasks, where the number of available data sources and their associated heterogeneity drastically increases. In this scenario, where data is reused and repurposed on a new scale, the pattern expressed as Extract-Transform-Load (ETL) emerges as a fundamental and recurrent process for both producers and consumers of data on the Web. In addition to ETL, provenance, the representation of source artifacts, processes and agents behind data, becomes another cornerstone element forWeb data management, playing a fundamental role in data quality assessment, data semantics and facilitating the reproducibility of data transformation processes. This paper proposes the convergence of these two Web data management concerns, introducing a principled provenance model for ETL processes in the form of a vocabulary based on the Open Provenance Model (OPM) standard and focusing on the provision of an interoperable provenance model for ETL environments. The proposed ETL provenance model is instantiated in a real-world sustainability reporting scenario.

KW - Data transformation

KW - ETL

KW - Linked Data · Web

KW - Provenance

UR - https://www.scopus.com/pages/publications/84942568704

U2 - 10.1007/978-3-662-46641-4_4

DO - 10.1007/978-3-662-46641-4_4

M3 - Conference Publication

AN - SCOPUS:84942568704

SN - 9783662466407

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 43

EP - 57

BT - The Semantic Web

A2 - Passant, Alexandre

A2 - Norton, Barry

A2 - Valle, Emanuele Della

A2 - Troncy, Raphael

A2 - Fundulaki, Irini

A2 - Simperl, Elena

A2 - Mladenic, Dunja

PB - Springer-Verlag

T2 - Extended Semantic Web Conference, ESWC 2012

Y2 - 27 May 2012 through 31 May 2012

ER -

Freitas A, Kömpgen B, Oliveira JG, O’Riain S, Curry E. Representing interoperable provenance descriptions for ETL workflows. In Passant A, Norton B, Valle ED, Troncy R, Fundulaki I, Simperl E, Mladenic D, editors, The Semantic Web: ESWC 2012 Satellite Events - Revised Selected Papers. Springer-Verlag. 2015. p. 43-57. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-662-46641-4_4

Representing interoperable provenance descriptions for ETL workflows

Abstract

Publication series

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Fingerprint

Cite this