Licence to predict – Investigating approaches to modelling low-occurrence deep-sea Irish Antipatharia with a new evaluation metric

Species distribution modelling has proven effective in locating cold-water coral hotspots in the deep sea. Maximum entropy (MaxEnt) and gradient boosting (GBM) are equally good at modelling rare species, but previous efforts to predict the distribution of deep-sea Antipatharia have mostly employed MaxEnt. This study investigates how algorithm choice (MaxEnt or GBM) and the application of backward stepwise variable pre-selection influence model performance and generalisation. Thirty-six models (four frameworks for nine black coral morphospecies) were built and evaluated using AUC, TSS, and the average TSS index (ATI), a novel metric that calculates TSS drops between training and test data to measure generalisation capability. ATI identified concerns in model generalisation that were not captured by AUC and TSS, making it a valuable tool for evaluating predictive model quality in conservation applications. MaxEnt outperformed GBM in predicting black coral distribution, and the pre-selection of variables did not improve performance. Satisfactory results were only obtained for the two MaxEnt models of Stichopathes gravieri, providing insights into the implications of using small datasets in conservation efforts. The inconsistencies in our findings do not lead us to recommend the use of a single model type in future studies; rather, we stress the importance of careful evaluation of model metrics with ecological knowledge of the species distribution when applying a dataset to conservation purposes.