NUIG-DSIs submission to The GEM Benchmark 2021

Nivranshu Pasricha; Mihael Mihael; Paul Buitelaar

doi:10.13025/21350

NUIG-DSIs submission to The GEM Benchmark 2021

Nivranshu Pasricha
, Mihael Mihael
, Paul Buitelaar

Computer Science

Research output: Chapter in Book or Conference Publication/Proceeding › Conference Publication › peer-review

Abstract

This paper describes the submission by NUIG-DSI to the GEM benchmark 2021. We participate in the modeling shared task where we submit outputs on four datasets for data-to-text generation, namely, DART, WebNLG (en), E2E and CommonGen. We follow an approach similar to the one described in the GEM benchmark paper where we use the pre-trained T5-base model for our submission. We train this model on additional monolingual data where we experiment with different masking strategies specifically focused on masking entities, predicates and concepts as well as a random masking strategy for pre-training. In our results we find that random masking performs the best in terms of automatic evaluation metrics, though the results are not statistically significantly different compared to other masking strategies.

Original language	English (Ireland)
Title of host publication	Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)
Place of Publication	Online
Publisher	Association for Computational Linguistics
DOIs	https://doi.org/10.13025/21350 https://doi.org/10.18653/v1/2021.gem-1.13
Publication status	Published - 1 Aug 2021

Keywords

Data Science Institute

Access to Document

10.13025/21350Licence: CC BY
10.18653/v1/2021.gem-1.13

2021Accepted author manuscript, 241 KBLicence: CC BY

Cite this

@inproceedings{317c2ed773d84d8ba0b5cad77ef356c1,

title = "NUIG-DSIs submission to The GEM Benchmark 2021",

abstract = "This paper describes the submission by NUIG-DSI to the GEM benchmark 2021. We participate in the modeling shared task where we submit outputs on four datasets for data-to-text generation, namely, DART, WebNLG (en), E2E and CommonGen. We follow an approach similar to the one described in the GEM benchmark paper where we use the pre-trained T5-base model for our submission. We train this model on additional monolingual data where we experiment with different masking strategies specifically focused on masking entities, predicates and concepts as well as a random masking strategy for pre-training. In our results we find that random masking performs the best in terms of automatic evaluation metrics, though the results are not statistically significantly different compared to other masking strategies.",

keywords = "Data Science Institute",

author = "Nivranshu Pasricha and Mihael Mihael and Paul Buitelaar",

year = "2021",

month = aug,

day = "1",

doi = "10.13025/21350",

language = "English (Ireland)",

booktitle = "Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)",

publisher = "Association for Computational Linguistics",

}

NUIG-DSIs submission to The GEM Benchmark 2021. / Pasricha, Nivranshu; Mihael, Mihael; Buitelaar, Paul.
Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021). Online: Association for Computational Linguistics, 2021.

Research output: Chapter in Book or Conference Publication/Proceeding › Conference Publication › peer-review

TY - GEN

T1 - NUIG-DSIs submission to The GEM Benchmark 2021

AU - Pasricha, Nivranshu

AU - Mihael, Mihael

AU - Buitelaar, Paul

PY - 2021/8/1

Y1 - 2021/8/1

N2 - This paper describes the submission by NUIG-DSI to the GEM benchmark 2021. We participate in the modeling shared task where we submit outputs on four datasets for data-to-text generation, namely, DART, WebNLG (en), E2E and CommonGen. We follow an approach similar to the one described in the GEM benchmark paper where we use the pre-trained T5-base model for our submission. We train this model on additional monolingual data where we experiment with different masking strategies specifically focused on masking entities, predicates and concepts as well as a random masking strategy for pre-training. In our results we find that random masking performs the best in terms of automatic evaluation metrics, though the results are not statistically significantly different compared to other masking strategies.

AB - This paper describes the submission by NUIG-DSI to the GEM benchmark 2021. We participate in the modeling shared task where we submit outputs on four datasets for data-to-text generation, namely, DART, WebNLG (en), E2E and CommonGen. We follow an approach similar to the one described in the GEM benchmark paper where we use the pre-trained T5-base model for our submission. We train this model on additional monolingual data where we experiment with different masking strategies specifically focused on masking entities, predicates and concepts as well as a random masking strategy for pre-training. In our results we find that random masking performs the best in terms of automatic evaluation metrics, though the results are not statistically significantly different compared to other masking strategies.

KW - Data Science Institute

UR - http://hdl.handle.net/10379/16886

U2 - 10.13025/21350

DO - 10.13025/21350

M3 - Conference Publication

BT - Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021)

PB - Association for Computational Linguistics

CY - Online

ER -

NUIG-DSIs submission to The GEM Benchmark 2021

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this