Presentation
34. Evaluation Approaches for Human-AI Teaming: Guidance for Practitioners
SessionPoster Session 2
DescriptionAre you a practitioner who needs to understand how well human-AI teaming is supported in your tool? This poster provides actionable lessons learned, level of effort, and appropriate development lifecycle phase for a range of user engagement methods. We describe three types of user engagements: Co-design with subject matter expert (SME) teammates, Roundtables with a larger group of SMEs, and a human-in-the-loop experiment. We present the metrics and techniques used and the insights they provided about human-AI teaming. This human-AI teaming research was conducted in the domain of the FAA’s Traffic Flow Management (TFM). The problem dynamics are complex; the forecast uncertainties are high; and the final decisions are made through a de-centralized, multi-stakeholder, collaborative process. As such, it is not sufficient to build an AI solution. The AI automation must be designed to partner with human collaborators so that the TFM planners trust it appropriately and use it effectively.
Event Type
Poster
TimeThursday, September 12th5:30pm - 6:30pm MST
LocationMcArthur Ballroom
Aging
Augmented Cognition
Children's Issues
Communications
Cybersecurity
Education
Environmental Design
General Sessions
Human AI Robot Teaming (AI)
Macroergonomics
Occupational Ergonomics
Student Forum
Surface Transportation
Sustainability
System Development