Understanding Challenges and Needs of Using AI in Web Automation Systems

Abstract

Web automation—the process of simulating user interactions with websites—has gained significant attention due to advances in Artificial Intelligence and increased digitization of services. While web automation offers potential benefits by streamlining tasks and addressing accessibility barriers, it also introduces unique challenges due to the complexities of automating interactions with interfaces designed for humans. Beyond technical advancements, addressing these challenges requires meeting human-centric needs and concerns surrounding web automation. In this paper, we explore critical questions about users’ web automation needs and preferences through a series of interviews with a diverse group of participants, including individuals across various age ranges and users with visual impairments. Our findings offer insights on how users weigh concerns such as privacy, error rates, efficiency, and usefulness in deciding what level of automation might be appropriate for a given task. We highlight critical areas for improvement inautomation tools and design principles for future systems.