Applying Multi-Agent Reinforcement Learning to Watershed Management

Multi-Agent Reinforcement Learning (MARL) is an area of research that combines Reinforcement Learning (RL) with Multi-Agent Systems (MAS). In MARL, agents learn over time by trial and error, what actions to take depending on the state of the environment. The focus of this paper will be to apply MARL to the Watershed management problem. This problem is complex due to the constrained nature of it. The problem consists of a series of interested parties, all seeking to withdraw water from a river in order to irrigate farms, supply a city and produce energy. Enough water must also be available for the surrounding ecosystem. In this paper, the problem is defined and tailored to the MARL algorithm by discretizing the problem space. In order to gauge the performance of the MARL algorithm, it is then also compared to a state of the art heuristic optimisation algorithm, Particle Swarm Optimisation (PSO). The results of this paper reveal that the MARL algorithm can consistently produce valid solutions however it performs worse on average than the PSO. Interestingly, the granularity of the problem is not the reason for the sub optimal performance of MARL. In some cases the performance of MARL is nearly as good as the PSO, and in terms of convergence, MARL performs better.