Robots Controlled by Motor Imagery Brain-Computer Interfaces: A Brief Overview
Brain-computer interface (BCI) is a revolutionary technology that establishes a direct communication channel between the brain and external devices. By converting neural signals into actionable commands, BCI allows for seamless interaction with connected devices. This versatile technology has found applications in various fields, including assistance for individuals with disabilities, virtual reality systems, neurogaming experiences, cognitive enhancement techniques, and military operations.
Figure 1 How generally an MI-BCI works.
In this paper, we will delve into the concept of Motor Imagery (MI), which involves mentally simulating specific movements without physically executing them. MI enables individuals to engage in mental training exercises where they can visualize and prepare for physical actions. This process aids in rehabilitating weakened neural pathways among elderly populations while also serving as a tool for athletes to enhance their performance through mental conditioning. The potential applications of MI span across medical treatments and athlete development programs.
Figure 2 Detailed flowgraph of MI-BCI systems.
The inception of BCI owes much credit to Hans Berger, who made significant contributions by discovering the electrical activity within the human brain and inventing equipment capable of recording EEG signals. Although Berger’s initial tape recorder was relatively primitive compared to modern standards, his pioneering work laid down the fundamental groundwork for utilizing EEG data as a means to control external devices effectively. Almost four decades after Berger’s breakthroughs were established, Alvin Lucier took an intriguing step forward when he attempted to play percussion music solely using his thoughts during one notable concert hall performance. This innovative attempt, although not reaching musical satisfaction, marked an essential milestone in advancing BCI technology. The term “BCI” was first coined by Prof. Jacques Vidal in 1977, establishing him as the pioneer behind this groundbreaking technology. In another significant advancement, Steve Bozinovski successfully engineered a robot that could be controlled through non-invasive EEG signals, initially limited to basic commands such as starting and stopping. This achievement sparked widespread interest among scientists and investors alike, leading to greater exploration and investment in BCI research. Figure 3 shows the mentioned timeline in detail.
Figure 3 The timeline of BCI progress.
Evocation Paradigms
- Single MI-Paradigm:
The MI-BCI systems achieve user control by identifying and quantifying the brain signals associated with the user’s movement intentions and converting them into control instructions. Typically, these systems comprise three components: signal acquisition, signal processing, and output equipment. The initial instance of a single MI-BCI system controlling a robot dates back to 2005. The devised system interpreted EEG signals generated from the user’s mental imagery of left or right limb movement as commands to steer an electric wheelchair left or right directly. (Figure 4)
Figure 4 Translated the EEG signals generated by users’ brain imagination on the movement of the left or right limb.
Subsequently, Choi et al. developed a brain-controlled wheelchair robot based on motor imagery BCI, capable of executing not only left and right movements but also forward commands. Akce and colleagues utilized pilot EEG signals to control an unmanned aircraft at a fixed altitude remotely. Chae et al. explored a humanoid robot controlled by an MI-BCI system, analyzing EEG amplitude characteristics through power spectrum analysis and selecting information based on Fisher’s ratio. Subjects controlled the humanoid robot to navigate an indoor maze using real-time images from the robot’s head cameras and EEG signals.
Figure 5 Humanoid robot controlled by an MI-BCI system.
Lafleur et al. designed a brain-controlled quadcopter with a linear classifier classifying tasks, achieving movements like lifting, landing, left, right, forward, and backward. Real-time visual feedback from a forward camera enabled precise control, resulting in a design success rate of 90.5%. Vijayendra et al. employed an artificial neural network to classify EEG data, enabling translation and angular velocity control of a four-axis unmanned aerial vehicle.
Figure 7 Finger robotic exoskeleton;
Plechawska et al. developed a direct BCI for controlling a robotic arm, despite low EEG signal classification accuracy. Norman et al. designed a finger robotic exoskeleton for rehabilitation therapy, where unimpaired subjects used the robot to play a rehabilitation-focused computer game.
Figure 1 Stages of embryonic and postnatal brain maturation.
Liu et al. proposed a dual-arm robot controlled remotely through a brain-machine interface, using support vector machines based on the ERD/ERS phenomenon to identify the operator’s intentions and control both arms of the robot. In an experiment involving five volunteers, all successfully directed a robotic arm to lift and lower a box using their brain signals.
Figure 8 Remotely controlled dual-arm robot.
- Hybrid Paradigms:
Hybrid paradigms involve combining EEG signals with other modalities in BCI systems, such as electromyography, functional near-infrared spectroscopy, and electrooculography, to increase control instructions and enhance classification accuracy. For instance, Fazli et al. introduced a hybrid paradigm combining EEG and infrared spectroscopy, demonstrating improved classification and recognition accuracy for MI tasks. In another project, a system combining EEG and EOG was designed to control a quadcopter. Two electrodes around the subject’s eye tracked eye muscle activity, significantly enhancing the accuracy of EEG signal classification.
Feature Extraction Methods:
Feature extraction plays a crucial role in signal processing within the context of MI-BCI systems. Its primary objective is to convert pre-processed signals into feature vectors, eliminate unnecessary data within these vectors, and emphasize the crucial features of the pre-processed signals. In the realm of motor imagery tasks, feature extraction primarily targets the frequency domain and spatial information. Common techniques encompass the Fourier transform, auto-regressive models, wavelet transform, common space pattern, and others. Fourier transform, which involves Fast Fourier transform and discrete Fourier transform, is a key method. Fast Fourier transform outshines the discrete Fourier transform due to its simplicity in calculation and shorter processing time, rendering it widely adopted. A study by Sheeded et al. devised a robotic arm and applied FFT to extract EEG signal features from tasks such as arm closure, arm extension, and hand holding. Wavelet transform, known for its flexible time-frequency resolution, employs variable time and frequency windows to progressively refine signals. It enables the simultaneous expression of signal energy intensity or density in both the time and frequency domains. Another influential factor is the Principal Component Analysis (PCA) algorithm, which aims to reduce data dimensionality. It transforms a set of interconnected features into a comprehensive set without correlations, using orthogonal vectors. The resulting new features are linear combinations of the original features, arranged in descending order of variance. The Common Spatial Pattern (CSP) algorithm is widely utilized to enhance MI classification performance. It furnishes a set of spatial filters that diagonalize the covariance matrix of data from two types, thereby extracting distinguishable features that represent a significant portion of the EEG signal information. Comparing various feature extraction methods, the CSP algorithm, along with its enhanced versions, stands out in maximizing signal differences across different MI tasks. Its widespread use demonstrates its efficacy in improving MI classification performance. This filtering method exhibits strong separation characteristics, effectively extracting distinct components associated with different tasks in EEG signals.
Applications:
The MI-BCI system has garnered extensive attention due to its potential applications in motor control, neurological rehabilitation, and intelligent operation in specialized environments. Depending on the application context, robots controlled by MI-BCI systems can be categorized into motion control types and medical rehabilitation assistance types. Notably, there is a growing interest and investment in the military sector globally, with the U.S. Naval Institute recognizing BCI as a transformative technology, deeming it a “game changer”. Their perspective emphasizes the significance of cognitive skills over equipment and weaponry in combat situations. A notable project at the American Naval Institute focuses on enhancing the OODA (Observation, Orientation, Decision, Action) loop through BCI technology. The OODA loop involves distinct phases, starting with observation, where the pilot collects data from all available senses, followed by orientation, where the pilot analyzes and synthesizes the gathered data. The subsequent decision phase involves determining a course of action, and finally, the action phase entails realizing the decision through physical action. Importantly, each stage of the OODA loop provides an opportunity for reassessment as feedback is observed. Time emerges as a critical parameter influencing combat outcomes, underscoring the importance of rapid action. In tactical engagements, those who navigate the OODA loop swiftly gain an advantage, potentially prevailing over opponents with superior weapons and larger forces. This strategic advantage enables savvy combatants to outthink, outmaneuver, and overcome formidable adversaries by effectively navigating and disrupting their opponent’s OODA loop.
Summary and Future Directions:
This paper provides a thorough examination of the progress in the field of robots controlled by motor imagery EEG signals. Our study conducts a more comprehensive exploration of robots operating in diverse scenarios under the influence of motor imagery EEG signals. We particularly emphasize the development and trajectory of hybrid paradigms. Our perspective suggests that the control of external devices within MI-BCI systems has transitioned from singular motor imagery evocation paradigms to hybrid paradigms. This evolution enhances the range of control commands for robots, diversifying their functionalities. Currently, MI-BCI technology is undergoing significant advancement. In the areas of signal processing and asynchronous control research, numerous issues and challenges persist. In future investigations, the integration of hybrid paradigms and the increasing sophistication of deep learning are expected to substantially enhance the overall capabilities of MI-BCI-controlled robot systems. A prominent research focus will be on rehabilitation robots utilizing MI training to facilitate the recovery of upper limb motor function in stroke patients. The introduction of innovative hybrid paradigms is anticipated to amplify patient engagement, evoke intentional movement, and enhance the effectiveness of robot-assisted rehabilitation.
Author: Amirhossein Maddah
References:
[1] Vahid Bastani. treatta. In: website, Treatta (2020), https://treatta.com/what-is–bci/.
[2] Benjamin Blankertz et al. The Berlin Brain-Computer Interface: Progress Beyond Communication and Control. In: Frontiers in Neuroscience 10 (Nov. 2016). ISSN: 1662-453X. DOI: 10.3389/fnins.2016.00530.
[3] F. Brocal. Brain-computer interfaces in safety and security
fields: Risks and applications. In: Safety Science 160 (Apr. 2023), p. 106051. ISSN: 0925-7535. DOI: 10.1016/j.ssci.2022.106051.
[4] Major Kevin S Woods. The Changing Application of Maneuver. In: Sta (1997).
[5] Jincai Zhang and Mei Wang. A survey on robots controlled by motor imagery brain-computer interfaces. In: Cognitive Robotics 1 (2021), pp. 12–24. ISSN: 2667-2413. DOI: 10.1016/j.cogr.2021.02.001.
Author: Amirhossein Maddah
