Harmony of Mind and Machine

The integration of external devices into human life poses challenges in establishing a sense of control and ownership. This study, conducted at Cross Labs, explores the nuances of control and ownership perception concerning an external device, essential for individuals fully dependent on it. The research examines combinations of control and ownership perceptions, delving into the user's agency and the feeling of possession over the device. Previous studies on the illusion and delusion of control, like the rubber hand experiment and the Ouija board phenomenon, guide this exploration.

Utilizing Unity and a touch haptic device, the study created controlled environments in which users interacted with a digital ball. Two scenarios were designed: one with user control and another introducing randomness. Results revealed that users exhibited longer perceived time and increased pen movement when control was lacking, indicating a loss of perceived agency. This intentional binding effect underscored the feeling of lost control. Further research aims to refine these findings by manipulating levels of control, paving the way for a comprehensive understanding of user-device interaction. Future work could extend the study to encompass varying levels of ownership, providing essential insights into user-device integration.

Introduction

When a human decides to integrate an external device into their own life, such as a robotic arm or computer interface, they often struggle with feeling true ownership over the device. This is especially important in the case of full dependence on the device to perform daily functions. The goal of my project at Cross Labs is to test the level of control and ownership one has over an external device, as well as the perception of these two classes, so that a user can transition into utilizing an external device more efficiently.

Though some of these concepts have been studied before, this project goes into extensive detail about how to map out the 16 combinations of control, perception of control, ownership, and perception of ownership and the lack thereof.

Though some of these concepts have been studied before, this project goes into extensive detail about how to map out the 16 combinations of control, perception of control, ownership, and perception of ownership and the lack thereof. It also looks to create a single series of tests in order to determine where the user feels that they still have a sense of control and agency over a device while there may still be some margin of error.

Regarding previous work, the illusion of control and agency has been extensively studied in works such as Madary (2022} and Wegner (2002). Some examples mentioned in these works that guide us in understanding the illusion of control and ownership include the rubber hand experiment and the concept of the Ouija board. In the rubber hand experiment, a participant has their hand out of sight and a rubber hand is placed in front of them. The researcher then interacts with the rubber hand and the participant's real hand at the same time by stroking them with a paintbrush and then ultimately hitting only the rubber hand with a hammer. The participant will often react dramatically and pull their real hand away, showing perceived ownership over the fake hand, while they may feel lack of ownership of their real hand temporarily. With the Ouija board, participants will all place their hands on a planchette and ask a question. The planchette will then move somewhere on the board, either to yes, no, or a letter to spell out a word. While participants believe that spirits are in control of moving the planchette, it is actually someone in the group moving it without realization. Here, they feel a lack of  perceived control while they actually do have control. We can use these ideas to begin to understand combinations of perceived control and ownership and lack thereof when determining how to design our experiments.

Methods

In order to create real-time experiments, I had to integrate software through the game engine Unity and hardware through a touch haptic device. The touch haptic device applies force feedback on the users' hand, allowing them to feel virtual objects and producing true-to-life touch sensations as users manipulate on-screen 3D objects. An image of it is below.

Because my goal was to use the device to move a ball with the pen, I wanted to initially load the pen in Unity and henceforth allow it to interact with a digital ball and box. First, I worked to connect the device in Unity, which I did by following the user manual, OpenHaptics Unity Plugin V2 – User Guide. I had downloaded the open haptic starter package, which allowed me to insert both the device and a virtual pen onto the screen.

Once I had everything loaded into Unity, I decided I wanted to make a stage with four surrounding walls, a ball to get pushed, and a hole in the ground for the user to push the ball into to indicate the end of the trial. To complete this, I simply used Unity's built-in shape features and dragged and manipulated different shapes into the following setup described above, with an image of that setup below. Additionally, I ended up adding a box on the corner of the arena for a user to hit to indicate the end of the trial.

Finally, I had to use C sharp in the code editor Visual Studios to manipulate the error of the ball when the user would hit it and record the distance the pen had moved total when the user would hit the box. In order to make the ball respond randomly to being hit by the pen, I added random force and error percentage components that can be changed by the user for each trial. In order to record the distance the pen had travelled from the beginning of the trial until it hit the box, I added code that could track and print the distance in which the ball moved before hitting the box. Both of these codes I attached to the ball, with the distance tracker linked to the 'Box' tag. The code is linked and detailed in my Github.

Results

I ultimately ended up creating two scenarios for the ball, one in which the user had complete control over the direction the ball is being hit and another where the ball seemingly reacts randomly when hit by the user. I intend on continuing work in this portion where randomness on the ball can be varied on a sliding scale to test in percentages of 10 to more closely identify at what point the user loses control.

I ran several tests with participants from the lab where they would try to move the ball into the hole in both control environments after a few rounds of practice. Below I have an image of what the environment in Unity looked like and the table where I listed out these trial results.

Discussion

The results from this trial align with several ideas that lack of control can be perceived from several aspects of the trial without having to actually ask the user if they feel in control over the device. The first is that the pen will move a longer distance when the user does not feel control. By simply noting the distance levels in the table above, it is clear that distances substantially increased when the user had to chase the ball around and correct for randomness. While this will not always be the case due to the ball occasionally moving in randomly into the hole, it can be assumed to act accordingly the majority of the time.

We can observe that the user feels that the trials are longer when they do not have control, whether or not the trials are actually longer.

Next, we can observe that the user feels that the trials are longer when they do not have control, whether or not the trials are actually longer. This effect is called intentional binding Moore and Obhi (2012), or when an action follows an expected effect, time feels as though it passes quicker. A common example of this that many people can agree on is the idea that when you are driving to a place you have never been before, often the way back home will feel shorter. This is because the subjective perception of time slows down during unfamiliar experiences, or the simple idea that you feel a lack of control over where you are or what is to come.  

As aforementioned, this project is still in the beginning stages due to having only two scenarios modeling control and lack thereof. I have gotten initial data and set the experiment in motion, but being able to determine at what percentage a user actually feels loss of control would be integral to actually integrating these ideas to an external device. I had attempted different strategies of tweaking randomness and force elements within the code to make this happen, but unfortunately I just felt as though these changes didn't significantly impact how the ball moved.

Once different levels of control can be modeled, this project can begin to shift to model different levels of ownership. An idea for this would be that modeling longer tasks where the user is using the pen for an extended period of time and completing more complicated tasks. Similar to the rubber hand experiment Rohde et al. (2011), they will likely begin to feel greater ownership over the device through this extended period of use. The combination of varying levels of ownership and control can be then combined to model each of the 16 aforementioned scenarios and create an all encompassing series of tests to determine exact levels of error that can allow for a user to continue to feel that an external device is part of themselves.

Acknowledgements

I am thankful to all of my supervisors who were able to help me bounce around ideas on how we were going to approach the project and help me out with my many questions along the way. I am also thankful to the other interns I got to work with for inspiring me with how they approached all of their uniquely interesting projects and always keeping the environment fun.

References:

Madary, M. (2022). The illusion of agency in human–computer interaction. Neuroethics, 15(1):1–15.

Moore, J. and Obhi, S. (2012). Intentional binding and the sense of agency: a review. Consciousness and Cognition, 21(1):546–561.

Rohde, M., Di Luca, M., and O. Ernst, M. (2011). The rubber hand illusion: Feeling of ownership and proprioceptive drift do not go hand in hand. Plos One.

Wegner, D. M. (2002). The Illusion of Conscious Will. MIT press.