Brain Responses During Robot-Error Observation
Brain-controlled robots are a promising new type of assistive device for severely impaired
persons. Little is however known about how to optimize the interaction of humans and brain-controlled
robots. Information about the human’s perceived correctness of robot performance might provide a useful
teaching signal for adaptive control algorithms and thus help enhancing robot control. Here, we studied
whether watching robots perform erroneous vs. correct action elicits differential brain responses that can
be decoded from single trials of electroencephalographic (EEG) recordings, and whether brain activity
during human-robot interaction is modulated by the robot’s visual similarity to a human.
To address these topics, we designed two experiments. In experiment I, participants watched a robot arm
pour liquid into a cup. The robot performed the action either erroneously or correctly, i.e. it either spilled
some liquid or not. In experiment II, participants observed two different types of robots, humanoid and
non-humanoid, grabbing a ball. The robots either managed to grab the ball or not.
We recorded high-resolution EEG during the observation tasks in both experiments to train a Filter Bank
Common Spatial Pattern (FBCSP) pipeline on the multivariate EEG signal and decode for the correctness
of the observed action, and for the type of the observed robot. Our findings show that it was possible to
decode both correctness and robot type for the majority of participants significantly, although often just
slightly, above chance level. Our findings suggest that non-invasive recordings of brain responses elicited
when observing robots indeed contain decodable information about the correctness of the robot’s action
and the type of observed robot. Our study also indicates that, given the, so far, relatively low decoding
accuracies, either further improvements in non-invasive recording and analysis techniques or the
utilization of intracranial measurements of neuronal activity will be necessary for practical applications.