p300-based-brain-computer-interface-speller-performance-estimation

P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation

Performance estimation is a necessary step in the development and validation of Brain-Computer Interface (BCI) systems. Unfortunately, even modern BCI ...

For intracortical brain-machine interfaces (BMIs), action potential voltage waveforms are often sorted to separate out individual neurons. If these neurons contain independent tuning information, this process could increase BMI performance. However, the sorting of action potentials ('spikes') requires high sampling rates and is computationally expensive. To explicitly define the difference between spike sorting and alternative methods, we quantified BMI decoder performance when using threshold-crossing events versus sorted action potentials.

Comparison of spike sorting and thresholding of voltage waveforms for intracortical brain-machine interface performance.

Challenges in improving the performance of dexterous upper-limb brain-machine interfaces (BMIs) have prompted renewed interest in quantifying the amount and type of sensory information naturally encoded in the primary motor cortex (M1). Previous single unit studies in monkeys showed M1 is responsive to tactile stimulation, as well as passive and active movement of the limbs. However, recent work in this area has focused primarily on proprioception. Here we examined instead how tactile somatosensation of the hand and fingers is represented in M1.

Robust tactile sensory responses in finger area of primate motor cortex relevant to prosthetic control.

Brain Computer Interfaces (BCIs) offer restoration of communication to those with the most severe movement impairments, but performance is not yet ideal. Previous work has demonstrated that latency jitter, the variation in timing of the brain responses, plays a critical role in determining BCI performance. In this study, we used Classifier-Based Latency Estimation (CBLE) and a wavelet transform to provide information about latency jitter to a second-level classifier. Three second-level classifiers were tested: least squares (LS), step-wise linear discriminant analysis (SWLDA), and support vector machine (SVM). Of these three, LS and SWLDA performed better than the original online classifier. The resulting combination demonstrated improved detection of brain responses for many participants, resulting in better BCI performance. Interestingly, the performance gain was greatest for those individuals for whom the BCI did not work well online, indicating that this method may be most suitable for improving performance of otherwise marginal participants.

Enhancing P300-BCI performance using latency estimation.

Monitoring the sleep patterns of children with autism spectrum disorders (ASD) and understanding how sleep quality influences their daytime behavior is an important issue that has received very limited attention. Polysomnography (PSG) is commonly used as a gold standard for evaluating sleep quality in children and adults. However, the intrusive nature of sensors used as part of PSG can themselves affect sleep and is, therefore, not suitable for children with ASD. In this study, we evaluate an unobtrusive and inexpensive bed system for in-home, long-term sleep quality monitoring using ballistocardiogram (BCG) signals. Using the BCG signals from this smart bed system, we define "restlessness" as a surrogate sleep quality estimator. Using this sleep feature, we build predictive models for daytime behavior based on 1-8 previous nights of sleep. Specifically, we use two supervised machine learning algorithms namely support vector machine (SVM) and artificial neural network (ANN). For all daytime behaviors, we achieve more than 78% and 79% accuracy of correctly predicting behavioral issues with both SVM and ANN classifiers, respectively. Our findings indicate the usefulness of our designed bed system and how the restlessness feature can improve the prediction performance.

Smart bed based daytime behavior prediction in Children with autism spectrum disorder - A Pilot Study.

Affective brain-computer interfaces are a relatively new area of research in affective computing. Estimation of affective states can improve human-computer interaction as well as improve the care of people with severe disabilities. To assess the effectiveness of EEG recordings for recognizing affective states, we used data collected in our lab as well as the publicly available DEAP database. We also reviewed the articles that used the DEAP database and found that a significant number of articles did not consider the presence of the class imbalance in the DEAP. Failing to consider class imbalance creates misleading results. Further, ignoring class imbalance makes the comparison of the results between studies using different datasets impossible, since different datasets will have different class imbalances. Class imbalance also shifts the chance level, hence it is vital to consider class bias while determining if the results are above chance. To properly account for the effect of class imbalance, we suggest the use of balanced accuracy as a performance metric, and its posterior distribution for computing credible intervals. For classification, we used features from the literature as well as theta beta-1 ratio. Results from DEAP and our data suggest that the beta band power, theta band power, and theta beta-1 ratio are better feature sets for classifying valence, arousal, and dominance, respectively.

Affective brain-computer interfaces: Choosing a meaningful performance measuring metric.

P300-based Brain-Computer Interface (BCI) performance is vulnerable to latency jitter. To investigate the role of latency jitter on BCI system performance, we proposed the classifier-based latency estimation (CBLE) method. In our previous study, CBLE was based on least-squares (LS) and stepwise linear discriminant analysis (SWLDA) classifiers. Here, we aim to extend the CBLE method using sparse autoencoders (SAE) to compare the SAE-based CBLE method with LS- and SWLDA-based CBLE. The newly-developed SAE-based CBLE and previously used methods are also applied to a newly-collected dataset to reduce the possibility of spurious correlations. Our results showed a significant (p<0.001) negative correlation between BCI accuracy and estimated latency jitter. Furthermore, we also examined the effect of the number of electrodes on each classification technique. Our results showed that on the whole, CBLE worked regardless of the classification method and electrode count; by contrast the effect of the number of electrodes on BCI performance was classifier dependent.

A Comparison of Classification Techniques to Predict Brain-Computer Interfaces Accuracy Using Classifier-Based Latency Estimation.

The goal of this work was to create a sharable dataset of heart-driven signals, including ballistocardiograms (BCGs) and time-aligned electrocardiograms (ECGs), photoplethysmograms (PPGs), and blood pressure waveforms.

Bed-Based Ballistocardiography: Dataset and Ability to Track Cardiovascular Parameters.

Eye Movement Desensitization and Reprocessing (EMDR) has demonstrated efficacy in treating major depressive disorder. EMDR increases cerebral perfusion in the anterior cingulate cortex (ACC) and dorsolateral prefrontal cortex (dlPFC). Activity in the ACC and dlPFC can be measured by theta cordance (TC) but has not been examined in EMDR. Ten participants (3 men, 7 women, M age = 42.31 ± 15.03) received ten 75 ± 15 minute EMDR sessions over 6.5 ± .5 weeks. Results indicated that PHQ-9 depression scores reduced from T1 (M = 13.9 ± 3.31) to T11 (M = 6.30 ± 3.23) with EMDR (SMD = 2.30), and that fTC but not pfTC was significantly related to this change. Depression declined as fTC declined. EMDR may engage the dlPFC or ACC that modulates depression and aid in reducing fTC and thus depression levels.

Clinical efficacy of EMDR in unipolar depression: Changes in theta cordance.

Brain-Computer Interfaces (BCIs) have been used to restore communication and control to people with severe paralysis. However, non-invasive BCIs based on electroencephalogram (EEG) are particularly vulnerable to noise artifacts. These artifacts, including electro-oculogram (EOG), can be orders of magnitude larger than the signal to be detected. Many automated methods have been proposed to remove EOG and other artifacts from EEG recordings, most based on blind source separation. This work presents a performance comparison of ten different automated artifact removal methods. Unfortunately, all tested methods substantially and significantly reduced P3 Speller BCI performance, and all methods were more likely to reduce performance than increase it. The least harmful methods were titled SOBI, JADER, and EFICA, but even these methods caused an average of approximately ten percentage points drop in BCI accuracy. Possible mechanistic causes for this empirical performance deduction are proposed.

Automated Artifact Rejection Algorithms Harm P3 Speller Brain-Computer Interface Performance.

Affective states play an important role in human behavior and decision-making. In recent years, several affective brain-computer interface (aBCI) studies have focused on developing an emotion classifier based on elicited emotions within the user. However, it is difficult to achieve consistency in elicited emotions across populations, which can lead to dataset imbalances. The experimental design presented in this paper seeks to avoid consistency issues by asking the participant to classify the emotion portrayed in images of facial expressions, rather than their own emotions. Priming is also a common technique used in psychology studies that is known to influence emotional perception. To improve participant accuracy, we investigated matching and mis-matched word priming for the facial expression images. Electro-encephalogram (EEG) data were used to generate images fed into a classifier based on the Big Transfer model, BiT-M R101x1. The primed images resulted in higher classification accuracy overall. Further, by building different classifier models for both mis-matched primed images and matching primed images, we were able to achieve classification accuracies above 90%. We also provided the classifier with the true labels of the photographs instead of the labels generated by the participants and achieved similar results. The experimental paradigm of measuring brain activity during the emotional classification of another individual provides consistently high, balanced classification accuracies.

The Effects of Word Priming on Emotion Classification from Neurological Signals.

Affective brain-computer interfaces are a fast-growing area of research. Accurate estimation of emotional states from physiological signals is of great interest to the fields of psychology and human-computer interaction. The DEAP dataset is one of the most popular datasets for emotional classification. In this study we generated heat maps from spectral data within the neurological signals found in the DEAP dataset. To account for the class imbalance within this dataset, we then discarded images belonging to the larger class. We used these images to fine-tune several Big Transfer neural networks for binary classification of arousal, valence, and dominance affective states. Our best classifier was able to achieve greater than 98% accuracy and 990% balanced accuracy in all three classification tasks. We also investigated the effects of this balancing method on our classifiers.

Applying Big Transfer-based classifiers to the DEAP dataset.

Heart rate variability (HRV) features support several clinical applications, including sleep staging, and ballistocardiograms (BCGs) can be used to unobtrusively estimate these features. Electrocardiography is the traditional clinical standard for HRV estimation, but BCGs and electrocardiograms (ECGs) yield different estimates for heartbeat intervals (HBIs), leading to differences in calculated HRV parameters. This study examines the viability of using BCG-based HRV features for sleep staging by quantifying the impact of these timing differences on the resulting parameters of interest. We introduced a range of synthetic time offsets to simulate the differences between BCG- and ECG-based heartbeat intervals, and the resulting HRV features are used to perform sleep staging. Subsequently, we draw a relationship between the mean absolute error in HBIs and the resulting sleep-staging performances. We also extend our previous work in heartbeat interval identification algorithms to demonstrate that our simulated timing jitters are close representatives of errors between heartbeat interval measurements. This work indicates that BCG-based sleep staging can produce accuracies comparable to ECG-based techniques such that at an HBI error range of up to 60 ms, the sleep-scoring error could increase from 17% to 25% based on one of the scenarios we examined.

Effects of Ballistocardiogram Peak Detection Jitters on the Quality of Heart Rate Variability Features: A Simulation-Based Case Study in the Context of Sleep Staging.

Article	Year
Comparison of spike sorting and thresholding of voltage waveforms for intracortical brain-machine interface performance. Journal of neural engineering\| PubMed ID: 25504690	2015
Robust tactile sensory responses in finger area of primate motor cortex relevant to prosthetic control. Journal of neural engineering\| PubMed ID: 28504971	2017
Enhancing P300-BCI performance using latency estimation. Brain computer interfaces (Abingdon, England)\| PubMed ID: 29725608	2017
Smart bed based daytime behavior prediction in Children with autism spectrum disorder - A Pilot Study. Medical engineering & physics\| PubMed ID: 32807344	2020
Affective brain-computer interfaces: Choosing a meaningful performance measuring metric. Computers in biology and medicine\| PubMed ID: 33007621	2020
A Comparison of Classification Techniques to Predict Brain-Computer Interfaces Accuracy Using Classifier-Based Latency Estimation. Brain sciences\| PubMed ID: 33066374	2020
Bed-Based Ballistocardiography: Dataset and Ability to Track Cardiovascular Parameters. Sensors (Basel, Switzerland)\| PubMed ID: 33383739	2020
Clinical efficacy of EMDR in unipolar depression: Changes in theta cordance. Psychiatry research\| PubMed ID: 33387752	2021
Automated Artifact Rejection Algorithms Harm P3 Speller Brain-Computer Interface Performance. Brain computer interfaces (Abingdon, England)\| PubMed ID: 34988241	2019
The Effects of Word Priming on Emotion Classification from Neurological Signals. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference\| PubMed ID: 36085617	2022
Applying Big Transfer-based classifiers to the DEAP dataset. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference\| PubMed ID: 36086186	2022
Effects of Ballistocardiogram Peak Detection Jitters on the Quality of Heart Rate Variability Features: A Simulation-Based Case Study in the Context of Sleep Staging. Sensors (Basel, Switzerland)\| PubMed ID: 36904896	2023

David E. Thompson

Biography

JoVE Journal Publications

Other Publications