Dynamic Thresholded Lexicographic Ordering

The goal of multi-objective problems is to find solutions that balance different objectives. When solving multi-objective problems using reinforcement learning linear scalarisation techniques are generally used, however system expertise is required to optimise the weights for linear scalarisation. Thresholded Lexicographic Ordering (TLO) is one technique that avoids the need for an expert to specify weights; instead a system designer can directly specify a preferred ordering over objectives, along with a desired threshold value for each objective. In this paper we propose a novel algorithm to dynamically set thresholds for use with TLO. We also present the first evaluation of TLO in a complex multi-objective multi-agent problem, the Dynamic Economic Emissions Dispatch domain. Our empirical results demonstrate that TLO with our dynamic thresholding algorithm achieves superior results when compared with a hand-tuned linear scalarisation method from previously published work.