Presentation
19. Investigating the Impact of User Interface Designs on Expectations About Large Language Models’ Capabilities
SessionPoster Session 1
DescriptionLarge Language Models (LLMs) with their novel conversational interaction format could create incorrectly calibrated expectations about their capabilities. The present study investigates human expectations towards a generic LLM’s capabilities and limitations. Participants of an online study were shown a series of prompts that cover a wide range of tasks and asked to assess the likelihood of the LLM being able to help with those tasks. The result is a catalog of people’s general expectations of LLM capabilities across various task domains. Depending on the actual capabilities of a specific system, this could inform developers of potential over- or under-reliance on this technology due to these misconceptions. To explore a potential way of correcting misconceptions we also attempted to manipulate their expectations with three different interface designs. In most of the tested task domains, such as computation and text processing, however, these seem to be insufficient to overpower people’s initial expectations.
Event Type
Poster
TimeWednesday, September 11th5:30pm - 6:30pm MST
LocationMcArthur Ballroom
Aerospace Systems
Cognitive Engineering & Decision Making
Computer Systems
Forensics Professional
Health Care
Human Performance Modeling
Individual Differences in Performance
Perception and Performance
Product Design
Safety
Training
Usability and System Evaluation
Extended Reality