Lead User Experience Researcher, Facebook (Meta)

Abstract: Artificial intelligence (AI) is increasingly considered a critical computational design material in developing innovative products, systems, and services. The framing of AI as a possible key constituent in the design process necessitates rethinking the end-goal function and use of computational design solutions in an era of “evolving complementary capabilities and doings” AI. In particular, machine learning has the potential to radically reorient the researcher and designer’s approach to crafting high-quality user experience as a human-centered design stance might now also have to accommodate the needs of machines.

One way to conceptualize how computational designers and UX researchers might handle AI as an interactive material is to focus initially on defining the desired outcome. If, for example, the outcome is to automate a task within a human replacement paradigm or to locate a photo within a transactional paradigm, the design constraints are relatively narrow. However, if the desired outcome is more aspirational, such as the parallel growth of humans and machines to improve the human condition, the design constraints and the design process will be quite different.

Education, healthcare, and social justice are three significant global areas of inquiry that could greatly benefit from ethical innovations in artificial intelligence. The worldwide COVID-19 pandemic brings these issues even more to the fore as recurring lockdowns keep people at home, necessitating the transfer of services to virtual domains. Over the last decade, important progress has been made within the area of telehealth and telemedicine in delivering healthcare in the community and at home at scale. As the global population ages, there is a growing need for rehabilitation services for debilitating illnesses such as stroke, Parkinson’s, and arthritis. Technology-assisted rehabilitation, using commercial consumer tools (e.g., Fitbit) or custom-designed products, shows promise, but issues of cost, patient acceptance, and replicating the expertise of the clinician present challenges. Effective, innovative rehabilitation in the clinic and at home requires that therapy assessment, training sessions, and activities of daily living are captured and analyzed in a manner that effectively supports evidence-based supervision and adaptation of therapy.

This talk takes a collaborative and human-centric computational design approach toward understanding the application of Human-Computer-Interaction (HCI) for complex Machine Learning (ML) in Embodied Learning (EL) scenarios. The work described in this document has focused on applying HCI methodologies to complex ML contexts to capture and assess how humans move, think, and learn in embodied spaces. HCI and ML can be considered existentially different in their primary objectives, methodologies, and evaluation processes within embodied spaces. HCI typically takes a human-centered approach, focusing on using human intelligence to best design solutions. In contrast, ML processes typically focus on using highly standardized, quantifiable datasets as input for extremely fast processors to provide generalizable outputs.

Human learning in embodied learning spaces is often considered tacit and challenging to uncover. This talk takes a three-pronged approach to designing a solution to reveal and augment this form of human knowledge. The first design challenge aims to understand how HCI and design can reveal and transform tacit human knowledge into explicit knowledge. The second challenge is transforming explicit human expertise into a computable model understandable to an AI. The third and final design challenge is to leverage AI computations to promote further learning and enhance human intelligence in embodied learning spaces The primary objective of my dissertation is to create an integrative HCI method that synergizes human intelligence with computational intelligence.

This talk discusses the methodology of Z methodology. This methodology comprises three highly interrelated and iterative processes: First, it transforms tacit human knowledge into explicit knowledge. Next, it converts explicit human knowledge into a computable model. Finally, it uses computational power to empower humans. To assess the validity of the Z methodology, I interrogated the approach within two critical healthcare frameworks: stroke rehabilitation movement training and stroke movement assessment.