SemR-11: A Multi-Lingual Gold- Standard for Semantic Similarity and Relatedness for Eleven Languages

Siamak Barzegar; Brian Davis; MANEL ZARROUK; Siegfried Handschuh; André Freitas

doi:10.13025/20965

SemR-11: A Multi-Lingual Gold- Standard for Semantic Similarity and Relatedness for Eleven Languages

Siamak Barzegar
, Brian Davis
, MANEL ZARROUK
, Siegfried Handschuh
, André Freitas

Computer Science

Research output: Chapter in Book or Conference Publication/Proceeding › Conference Publication › peer-review

Abstract

This work describes SemR-11, a multi-lingual dataset for evaluating semantic similarity and relatedness for 11 languages (German, French, Russian, Italian, Dutch, Chinese, Portuguese, Swedish, Spanish, Arabic and Persian). Semantic similarity and relatedness gold standards have been initially used to support the evaluation of semantic distance measures in the context of linguistic and knowledge resources and distributional semantic models. SemR-11 builds upon the English gold-standards of Miller & Charles (MC), Rubenstein & Goodenough (RG), WordSimilarity 353 (WS-353), and Simlex-999, providing a canonical translation for them. The final dataset consists of 15,917 word pairs and can be used to support the construction and evaluation of semantic similarity/relatedness and distributional semantic models. As a case study, the SemR-11 test collections was used to investigate how different distributional semantic models built from corpora in different languages and with different sizes perform in computing semantic relatedness similarity and relatedness tasks.

Original language	English (Ireland)
Title of host publication	Language Resources and Evaluation Conference
Publisher	European Language Resources Association
DOIs	https://doi.org/10.13025/20965
Publication status	Published - 1 Jan 2018

Access to Document

10.13025/20965Licence: CC BY-NC-ND

Fc4fe4d44e6ebac4da89ebe61ae752b8fdcfAccepted author manuscript, 516 KBLicence: CC BY-NC-ND

Cite this

@inproceedings{9cc2dcaf3e5a46caa31147b44068e123,

title = "SemR-11: A Multi-Lingual Gold- Standard for Semantic Similarity and Relatedness for Eleven Languages",

abstract = "This work describes SemR-11, a multi-lingual dataset for evaluating semantic similarity and relatedness for 11 languages (German, French, Russian, Italian, Dutch, Chinese, Portuguese, Swedish, Spanish, Arabic and Persian). Semantic similarity and relatedness gold standards have been initially used to support the evaluation of semantic distance measures in the context of linguistic and knowledge resources and distributional semantic models. SemR-11 builds upon the English gold-standards of Miller \& Charles (MC), Rubenstein \& Goodenough (RG), WordSimilarity 353 (WS-353), and Simlex-999, providing a canonical translation for them. The final dataset consists of 15,917 word pairs and can be used to support the construction and evaluation of semantic similarity/relatedness and distributional semantic models. As a case study, the SemR-11 test collections was used to investigate how different distributional semantic models built from corpora in different languages and with different sizes perform in computing semantic relatedness similarity and relatedness tasks.",

author = "Siamak Barzegar and Brian Davis and MANEL ZARROUK and Siegfried Handschuh and Andr{\'e} Freitas",

year = "2018",

month = jan,

day = "1",

doi = "10.13025/20965",

language = "English (Ireland)",

booktitle = "Language Resources and Evaluation Conference",

publisher = "European Language Resources Association",

}

TY - GEN

T1 - SemR-11: A Multi-Lingual Gold- Standard for Semantic Similarity and Relatedness for Eleven Languages

AU - Barzegar, Siamak

AU - Davis, Brian

AU - ZARROUK, MANEL

AU - Handschuh, Siegfried

AU - Freitas, André

PY - 2018/1/1

Y1 - 2018/1/1

N2 - This work describes SemR-11, a multi-lingual dataset for evaluating semantic similarity and relatedness for 11 languages (German, French, Russian, Italian, Dutch, Chinese, Portuguese, Swedish, Spanish, Arabic and Persian). Semantic similarity and relatedness gold standards have been initially used to support the evaluation of semantic distance measures in the context of linguistic and knowledge resources and distributional semantic models. SemR-11 builds upon the English gold-standards of Miller & Charles (MC), Rubenstein & Goodenough (RG), WordSimilarity 353 (WS-353), and Simlex-999, providing a canonical translation for them. The final dataset consists of 15,917 word pairs and can be used to support the construction and evaluation of semantic similarity/relatedness and distributional semantic models. As a case study, the SemR-11 test collections was used to investigate how different distributional semantic models built from corpora in different languages and with different sizes perform in computing semantic relatedness similarity and relatedness tasks.

AB - This work describes SemR-11, a multi-lingual dataset for evaluating semantic similarity and relatedness for 11 languages (German, French, Russian, Italian, Dutch, Chinese, Portuguese, Swedish, Spanish, Arabic and Persian). Semantic similarity and relatedness gold standards have been initially used to support the evaluation of semantic distance measures in the context of linguistic and knowledge resources and distributional semantic models. SemR-11 builds upon the English gold-standards of Miller & Charles (MC), Rubenstein & Goodenough (RG), WordSimilarity 353 (WS-353), and Simlex-999, providing a canonical translation for them. The final dataset consists of 15,917 word pairs and can be used to support the construction and evaluation of semantic similarity/relatedness and distributional semantic models. As a case study, the SemR-11 test collections was used to investigate how different distributional semantic models built from corpora in different languages and with different sizes perform in computing semantic relatedness similarity and relatedness tasks.

UR - http://hdl.handle.net/10379/14952

U2 - 10.13025/20965

DO - 10.13025/20965

M3 - Conference Publication

BT - Language Resources and Evaluation Conference

PB - European Language Resources Association

ER -

SemR-11: A Multi-Lingual Gold- Standard for Semantic Similarity and Relatedness for Eleven Languages

Abstract

Access to Document

Other files and links

Fingerprint

Cite this