Do AI Recommendation Systems Impede Personal Autonomy? A Philosophical Analysis

Abstract

AI recommendation systems (RecSys) are machine learning algorithms that suggest tailored content to users based on user data. They are increasingly implemented in search engines, social media sites, and various other online platforms, with the justification that it benefits both users and companies by enhancing user experience. Recently, some have argued that AI RecSys threaten personal autonomy by interfering with users’ decision-making and value-reflection processes. This removes users’ ability to independently form their own unique personal identity. My research challenges the reasoning of this view but not its conclusion. I argue that non-AI technologies that do not threaten personal autonomy also interfere with these identity-forming processes in similar ways. Instead, I argue that a different aspect, specific to AI RecSys, threatens personal autonomy in a much more subtle and nuanced way. That is, by building user profiles through machine learning algorithms, AI RecSys identify user preferences that users themselves may be unaware they have. RecSys then capitalize on these user preferences by suggesting content in line with RecSys goals based on these subconscious preferences. As users are unaware they have these preferences, they are unable to evaluate or change them, despite the fact that, if given the chance, users may wish to do so. Overall, RecSys identify subconscious user preferences and build highly detailed user profiles, using them to tailor user experience. In doing so, RecSys threaten personal autonomy by impeding users’ ability to reflect on and evaluate these preferences for themselves.