Attachment Figure Study 2: VR Conversation Partner
Working With:
Dr. Omri Gillath, Department of Psychology, University of Kansas
Dr, Michael Branicky, School of Engineering, University of Kansas
Dr, John Symons, Department of Philosophy, University of Kansas
Greg Thomas, Center for Design Research, University of Kansas
Dr. Alexander Schoemann, Department of Psychology, East Carolina University
Project Brief
I was asked to join a multi-disciplinary research team investigating how VR and AI might effect the ways that humans form close relationships. My task was to create a Virtual Reality environment in which a study participant could have a personal conversation with a simulated AI agent, along with a user interface for the experimenter to control the environment and the AI character during the conversation.
In the study, the participant sits in a room across from an animated virtual agent whom some of the subjects think is controlled by an AI system and other think is controlled by the researcher. The agent and participant take turns asking each other questions from a set of prompts and then listening to their partner’s responses and engaging in conversation. Half the participants are prompted with general conversational questions, while the other half are given more personal questions about their life and personal beliefs. As the conversations play out, the system is continuously measuring the distance between the subject and the agent, and how much the subject is looking directly at the agent - both signifiers of how engaged the subject is.
The aim is to see if there are differences in the ways that the participants engage with the virtual agent depending on if they think it’s an AI. Also, the study asks if engaging in a more personal conversation with the perceived AI changes the subject’s level of engagement.
Publication:
Submitted for NSF Grant Proposal: “Who can function as an attachment figure?”, 2023
Contributions:
3D modeling of virtual testing environment, Blender
3D animation of virtual characters, Blender
Programming of 3D game environment and testing application, Unity3D
The Setting
For many study participants this was the first time they would be experiencing VR, so it was important to provide a virtual environment that would put them at ease, and facilitate the conversation tasks of the study.
I constructed the room, modeled the seating, tables, and bookshelves with books, and finished out the room with stock models. I also textured the models and designed the lighting for the room.
To create the set, I studied other locations designed to put people at ease and facilitate conversations, finally honing in on the design of psychiatrist offices. Some notable features:
Seating:
Comfortable Seat - a large, high-baked chair gives the subject the feeling they can easily shift towards or away from the agent. A shoulder high back, and position against the back wall provides a sense of protection.
Clear Traffic Flow - the room layout provides the subject with a direct view of the door and a window, avoiding a sense of being trapped.
Light and Color - the room painted an warm neutral color accented with warm wooden furnishings creating a visual connection to a natural space of warmth and safety. Lights are also warm and softened, mimicking indirect sunlight, and placed near eye level to help illuminate faces evenly.
Balancing Clutter with Decor - room is furnished with a variety of non-distracting but cheerfully colored elements to make the space feel lived in. At the same time, the space is organized, avoiding clutter which could be hiding elements of danger.
Connecting Rug - a rug connecting the subject’s seating to the agent’s seating makes a visual connection between the two.
Safe Territory - placing small furniture such as side tables near the subject’s chair provides them personal territory, giving them a sense of an area of safety.
Research Sources:
Fang, Marie. “Therapist Office Interior Design on a Budget.” Private Practice Skills, 3 January 2019, https://privatepracticeskills.com/therapist-office-interior-design-budget/
Budds, Diana. “Home, soothing home: What science, color psychology, and interior designers tell us about creating a comforting space.” Curbd, 24 April 2020, https://archive.curbed.com/2020/4/24/21230141/comfortable-relaxing-soothing-home-interior-design
DeAngelis, Tori. “Healing by design: Nix the glass table and fill the room with light. These and other research-based design insights for therapists' offices can reap client benefits.” American Psychological Association, March 2017, Vol 48, No 3, pg 56, https://www.apa.org/monitor/2017/03/healing-design
Devlin, A. S., & Nasar, J. L. (2012). Impressions of psychotherapists' offices: Do therapists and clients agree?” Professional Psychology: Research and Practice, 43(2), 118–122. https://doi.org/10.1037/a0027292
Experimental Application
During the experiment, the researcher sits at a laptop in an adjoining room to the subject, and controls the progress of the experiment and the actions of the virtual agent. I designed the software and user interface, and coded the program using the Unity game engine.
The experimenter is a bit like a puppeteer. The actions available include:
facial expression - selection from one of 7 possible expressions for each model.
responsive movements - the agent can nod and shake their head, as well as tilt their head to one side or the other to reflect a sense of listening.
fidget actions - the agent can cycle through 4 different seated positions, animating between each to give the agent a more life like feeling over the course of the conversation.
text to speech - the system takes text input including pre-programmed prompts, a set of stock phrases, or directly typed chat input from the experimenter, and converts this into a computer voice for the agent to speak. The agent’s model automatically moves its mouth and lips based on the generated dialogue.
Agent Animation
I created 2 models to play the part of the AI agent, one female and one male which would be matched to the gender of the subject. These models were designed using ReadyPlayer.me, and then exported into Blender, where animation rigs were added.
There are a variety of stock character animations available, but most are designed for use in video games and so involve actions like running, jumping, shooting, etc. In order to convincingly participate in simple conversations it was necessary to animate a set of smaller more subtle motions and gestures. These included:
Sitting down in a chair.
Shifting through 4 different seating positions.
Making 3 different conversational glances
Nodding the head “yes” and shaking the head “no”.
Listening head tilts from side to side.
