Better GP benchmarks: Community survey results and proposals

David R. White; James McDermott; Mauro Castelli; Luca Manzoni; Brian W. Goldman; Gabriel Kronberger; Wojciech Jaśkowski; Una May O'Reilly; Sean Luke

doi:10.1007/s10710-012-9177-2

Better GP benchmarks: Community survey results and proposals

David R. White
, James McDermott
, Mauro Castelli
, Luca Manzoni
, Brian W. Goldman
, Gabriel Kronberger
, Wojciech Jaśkowski
, Una May O'Reilly
, Sean Luke

University of Glasgow
University College Dublin
Universidade Nova de Lisboa
University of Milano-Bicocca
Michigan State University
University of Applied Sciences for Health Professions Linz
Poznan Technical University
Massachusetts Institute of Technology
George Mason University

Research output: Contribution to a Journal (Peer & Non Peer) › Review article › peer-review

211 Citations (Scopus)

Abstract

We present the results of a community survey regarding genetic programming benchmark practices. Analysis shows broad consensus that improvement is needed in problem selection and experimental rigor. While views expressed in the survey dissuade us from proposing a large-scale benchmark suite, we find community support for creating a "blacklist" of problems which are in common use but have important flaws, and whose use should therefore be discouraged. We propose a set of possible replacement problems.

Original language	English
Pages (from-to)	3-29
Number of pages	27
Journal	Genetic Programming and Evolvable Machines
Volume	14
Issue number	1
DOIs	https://doi.org/10.1007/s10710-012-9177-2
Publication status	Published - Mar 2013
Externally published	Yes

Keywords

Benchmarks
Community survey
Genetic programming

Access to Document

10.1007/s10710-012-9177-2

Cite this

@article{c60bba94f20147439a1f7ad51b28f849,

title = "Better GP benchmarks: Community survey results and proposals",

abstract = "We present the results of a community survey regarding genetic programming benchmark practices. Analysis shows broad consensus that improvement is needed in problem selection and experimental rigor. While views expressed in the survey dissuade us from proposing a large-scale benchmark suite, we find community support for creating a {"}blacklist{"} of problems which are in common use but have important flaws, and whose use should therefore be discouraged. We propose a set of possible replacement problems.",

keywords = "Benchmarks, Community survey, Genetic programming",

author = "White, \{David R.\} and James McDermott and Mauro Castelli and Luca Manzoni and Goldman, \{Brian W.\} and Gabriel Kronberger and Wojciech Ja{\'s}kowski and O'Reilly, \{Una May\} and Sean Luke",

year = "2013",

month = mar,

doi = "10.1007/s10710-012-9177-2",

language = "English",

volume = "14",

pages = "3--29",

journal = "Genetic Programming and Evolvable Machines",

issn = "1389-2576",

publisher = "Springer",

number = "1",

}

TY - JOUR

T1 - Better GP benchmarks

T2 - Community survey results and proposals

AU - White, David R.

AU - McDermott, James

AU - Castelli, Mauro

AU - Manzoni, Luca

AU - Goldman, Brian W.

AU - Kronberger, Gabriel

AU - Jaśkowski, Wojciech

AU - O'Reilly, Una May

AU - Luke, Sean

PY - 2013/3

Y1 - 2013/3

N2 - We present the results of a community survey regarding genetic programming benchmark practices. Analysis shows broad consensus that improvement is needed in problem selection and experimental rigor. While views expressed in the survey dissuade us from proposing a large-scale benchmark suite, we find community support for creating a "blacklist" of problems which are in common use but have important flaws, and whose use should therefore be discouraged. We propose a set of possible replacement problems.

AB - We present the results of a community survey regarding genetic programming benchmark practices. Analysis shows broad consensus that improvement is needed in problem selection and experimental rigor. While views expressed in the survey dissuade us from proposing a large-scale benchmark suite, we find community support for creating a "blacklist" of problems which are in common use but have important flaws, and whose use should therefore be discouraged. We propose a set of possible replacement problems.

KW - Benchmarks

KW - Community survey

KW - Genetic programming

UR - https://www.scopus.com/pages/publications/84873706505

U2 - 10.1007/s10710-012-9177-2

DO - 10.1007/s10710-012-9177-2

M3 - Review article

SN - 1389-2576

VL - 14

SP - 3

EP - 29

JO - Genetic Programming and Evolvable Machines

JF - Genetic Programming and Evolvable Machines

IS - 1

ER -

Better GP benchmarks: Community survey results and proposals

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this