Vulnerable by Design: Reconsidering User Vulnerability and Recommender Systems

Recommender systems are invaluable in filtering vast amounts of information online. However, there are ethical challenges related to their objectives and design that have the potential to make some users vulnerable. Within emergent AI policy and regulation, vulnerable users have been given safeguarding measures to protect them against manipulation or exploitation. Vulnerable users are primarily defined as children and adults with particular characteristics. However, this definition focuses attention on the cause of vulnerability being the user’s characteristics rather than the design of recommender systems. However, all users regardless of personal characteristics, may be considered vulnerable to negative effects associated with recommender algorithms. This paper examines three threads of vulnerability within recommender systems: vulnerability derived from specific user characteristics, the vulnerabilities of the recommender systems themselves and vulnerability caused by the nature of interactions between users and recommendation algorithms. This paper argues that while it is essential to offer more protection and assistance to users who are considered vulnerable by virtue of certain characteristics, it is also important to acknowledge the possibility of all users being rendered vulnerable by features of the recommendation algorithms themselves. This reconsideration of the concept of vulnerability serves to highlight the importance of researching the effects of recommender algorithms on user groups that are currently understudied.