My research has two major strands: multisensory interaction, and skilled movement. Both strands are united by a focus on multisensory interaction, and theory translation: drawing on diverse theoretical resources and adapting them to the needs of design.
My work on multisensory interaction aims to bring greater specificity to UX research, and design for experience. My approach here is grounded in an understanding of sensory perception, affective science, and phenomenal experience. Much of my work in this area has focused on understanding synergies and interactions between different sensory modalities, and how these, and their affective associations, can be used to support design. In recent works, I have developed this line of research in both humanistic and engineering directions: I draw on classical Chinese poetry as a rich source of phenomenal description, constructed a theoretical framework for conceptualising the nuances and qualities of affective-thermal experiences. Based on this framework, I translated the theoretical account into a design toolkit with a purpose of engineering experience and technical affordances, which connects experiential qualities with concrete design parameters. This toolkit supports systematic design process for multisensory and affective technology. This process of theory-building and translation in my work has opened a rich seam of research ideas for me, my colleagues and students I am co-supervising.
My work on skilled movement is also grounded in understandings from psychology of multisensory perception - now to support skill learning, and the guidance of movement. During my PhD research I designed multisensory environments to support rehabilitation, wayfinding and sensory-motor skills. More recently I adapted methodologies from psychophysics to understand tactile skill in surgical training. Increasingly I have become interested in the complex contextual factors around skill learning and practice, and how these impact experience, outcomes, and research methods.
I am particularly proud of my recent works on thermal interaction, which connects perceptual science to aesthetic and humanistic accounts of experience. Over the years, generations of HCI researchers have returned to the question of how we should connect controlled scientific findings to the complexities of real-world interactions. In my recent work, I address this via a scientifically informed analysis of literary accounts of thermal-affective experience, following similar approaches in sensory psychology. These literary accounts provide eloquent and evocative descriptions of situated experience, situated in everyday life and activities - something otherwise difficult to capture. Analysing them allows me to suggest principles and guidelines for the design of thermalaffective experiences, and specify future empirical research in the field (The design framework paper). This work has opened a rich seam of research ideas for me which I am now addressing. This includes design research to understand the possibilities of thermal-haptic interfaces for affective regulation, and the development of a toolkit for prototyping which bridges experiential and engineering aspects of thermal-haptic experience (Thermal cards).
In the future, I would like to be known as a researcher who is leading the translational science in the field of affective interaction in HCI: my work creates bridges between experiential, scientific, and design approaches to support the development of multisensory technologies. My research toolkits will help engineering-focused researchers address the complexity of affective and situated experience, and help design-focused researchers connect their understandings to scientific and engineering principles. In particular, I plan to pursue this goal in the context of multisensory technologies for affective interaction and affective regulation in children and both neurodiverse and neurotypical populations.
Before my PhD, I studied Industrial design in the school of Mechenical Engineering (ME), Shandong University (SDU, Double-First Class University, Rank A）in China, with 1st class honours; and Interaction Design in the design school at Human University (HNU), one of the top 3 design schools in China.
As a designer, I designed the branding and visual identity system (VI system) for the China Academy Museum.
In previous intership/design projects, I’ve worked with Oulin and conducted ethnographic study for the design of IoT smart kitchen project.
I’ve participated in the creation of a campus journal “The voice of Yuelu” (Yuelu is a metaphor of academics) with the Business school in Hunan University. With the collaboration of MBA collegues, we together designed and edited the first five issues with focus on business models covering product desgin, interaction design and the general service industry (2013-2014).