Abstract
Due to the limited bandwidth and reliability of non-invasive electroencephalography (EEG)-based brain–computer interface (BCI) systems, it is difficult to control a six-degree-of-freedom (6-DoF) robotic arm using such a system. This thesis proposes a multimodal BCI system for controlling a robotic arm that divides control into three stages: initiating the task through right-hand motor imagery (MI), selecting an object of interest from a real scene using steady-state visual evoked potentials (SSVEPs) and webcam-based eye tracking, and autonomously performing the pick-and-drop task using a WidowX 250 robotic arm. Two separate offline experiments, each involving ten participants, were performed to test the SSVEP and MI components of the system. In the SSVEP experiment, several decoding pipelines were compared, and the effect of fusing eye tracking with SSVEPs was evaluated using different EEG window lengths. The TRCA-Net-Lite model with a gaze-attention module achieved a maximum accuracy of 98.00% using a 6.0-second window and a maximum information transfer rate of 48.01 bits/min using a 1.0-second window. In the MI experiment, several classifiers were compared for classifying right-hand MI, left-hand MI, and No-MI. DR-EEGNet achieved a three-class accuracy of 77.21% using subject-dependent four-fold cross-validation and was selected for online action initiation because of its compact structure. Online testing with six participants performing five trials each resulted in a first-attempt object-selection accuracy of 28/30 (93.3%). After the two incorrect selections were rejected using the double-blink mechanism and the object-selection stage was repeated, all 30 retained pick-and-drop tasks were successfully completed. The average task completion time was 49.52 ± 2.44 s. The main contributions of this thesis are a learned gaze-attention method for fusing SSVEP and webcam-derived gaze information, position-based selection that distinguishes visually identical objects, an MI-based arming gate for intentional task initiation, and a double-blink mechanism for resetting incorrect selections. This approach eliminates the need for continuous EEG-based movement control while retaining the user’s voluntary control over the task performed by the robotic arm.
Publication Date
8-2026
Document Type
Thesis
Student Type
Graduate
Degree Name
Electrical Engineering (MS)
Department, Program, or Center
Electrical Engineering
Advisor
Jinane Mounsef
Recommended Citation
Keshtkar, Mohammad, "Multimodal Brain-Computer Interface for Real-Scene Object Selection and Shared Robotic Arm Control" (2026). Thesis. Rochester Institute of Technology. Accessed from
https://repository.rit.edu/theses/12784
Campus
RIT Dubai
Comments
This thesis has been embargoed. The full-text will be available on or around 8/13/2027.