Abstract

MOTIVATION: Users must conform to the interfaces they are given, which are designed for a generalized user base rather than individual workflows or preferences. Existing UI customizations are often limited to fixed options predetermined by developers, restricting meaningful user control. Advances in generative AI introduce the possibility of modifying the underlying code of a production system that defines interfaces, enabling more flexible and targeted adaptations. However, the feasibility, scalability, and usability of such user-driven customization remain open questions. GOAL: The goal of this work is to help people customize their user interfaces, by investigating how they can linguistically specify the presentation and behavior of software interfaces beyond predefined design constraints METHODS: We developed GenUI, a generative AI system that enables users to modify web interfaces through natural language by generating and applying code-level changes. We conducted a user study to evaluate this approach in practice. Participants used GenUI to perform interface customization tasks, allowing us to assess task feasibility, cost, customization workflows, user opinion, failure modes, and sensitivity to randomness. RESULTS: Users were generally able to successfully perform interface modifications using natural language, with a median task success rate of 92%. GenUI enabled a wide range of customization types, including aesthetic, layout, content, and workflow changes, and led to an overall increase in perceived usability. However, modifications often involved substantial code changes, which did lead to the 8% failure rate (primarily type-related errors). Interaction costs were moderate, with noticeable latency and variability in generation time. CONCLUSIONS: Overall, GenUI is a promising approach for user-driven interface customization. However, current limitations in reliability, performance, and scalability suggest that further improvements in model specialization, system design, and computational infrastructure are necessary for practical deployment.

Publication Date

5-2026

Document Type

Thesis

Student Type

Graduate

Degree Name

Software Engineering (MS)

Department, Program, or Center

Software Engineering, Department of

College

Golisano College of Computing and Information Sciences

Advisor

Andy Meneely

Advisor/Committee Member

Kal Rabb

Campus

RIT – Main Campus

Download

Share

COinS