Updated on 2024-12-03

Brain

Publish Date	Title	Authors	URL	Abstract
2024-11-29	Dynamic EEG-fMRI mapping: Revealing the relationship between brain connectivity and cognitive state	Guiran Liu, Binrong Zhu	Link	This study investigated the dynamic connectivity patterns between EEG and fMRI modalities, contributing to our understanding of brain network interactions. By employing a comprehensive approach that integrated static and dynamic analyses of EEG-fMRI data, we were able to uncover distinct connectivity states and characterize their temporal fluctuations. The results revealed modular organization within the intrinsic connectivity networks (ICNs) of the brain, highlighting the significant roles of sensory systems and the default mode network. The use of a sliding window technique allowed us to assess how functional connectivity varies over time, further elucidating the transient nature of brain connectivity. Additionally, our findings align with previous literature, reinforcing the notion that cognitive states can be effectively identified through short-duration data, specifically within the 30-60 second timeframe. The established relationships between connectivity strength and cognitive processes, particularly during different visual states, underscore the relevance of our approach for future research into brain dynamics. Overall, this study not only enhances our understanding of the interplay between EEG and fMRI signals but also paves the way for further exploration into the neural correlates of cognitive functions and their implications in clinical settings. Future research should focus on refining these methodologies and exploring their applications in various cognitive and clinical contexts.
2024-11-29	Neuroplasticity and Psychedelics: a comprehensive examination of classic and non-classic compounds in pre and clinical models	Claudio Agnorelli, Meg Spriggs, Kate Godfrey, Gabriela Sawicka, Bettina Bohl, Hannah Douglass, Andrea Fagiolini, Hashemi Parastoo, Robin Carhart-Harris, David Nutt, David Erritzoe	Link	Neuroplasticity, the ability of the nervous system to adapt throughout an organism's lifespan, offers potential as both a biomarker and treatment target for neuropsychiatric conditions. Psychedelics, a burgeoning category of drugs, are increasingly prominent in psychiatric research, prompting inquiries into their mechanisms of action. Distinguishing themselves from traditional medications, psychedelics demonstrate rapid and enduring therapeutic effects after a single or few administrations, believed to stem from their neuroplasticity-enhancing properties. This review examines how classic psychedelics (e.g., LSD, psilocybin, N,N-DMT) and non-classic psychedelics (e.g., ketamine, MDMA) influence neuroplasticity. Drawing from preclinical and clinical studies, we explore the molecular, structural, and functional changes triggered by these agents. Animal studies suggest psychedelics induce heightened sensitivity of the nervous system to environmental stimuli (meta-plasticity), re-opening developmental windows for long-term structural changes (hyper-plasticity), with implications for mood and behavior. Translating these findings to humans faces challenges due to limitations in current imaging techniques. Nonetheless, promising new directions for human research are emerging, including the employment of novel positron-emission tomography (PET) radioligands, non-invasive brain stimulation methods, and multimodal approaches. By elucidating the interplay between psychedelics and neuroplasticity, this review informs the development of targeted interventions for neuropsychiatric disorders and advances understanding of psychedelics' therapeutic potential.
2024-11-29	The role of inhibitory neuronal variability in modulating phase diversity between coupled networks	Katiele V. P. Brito, Joana M. G. L. Silva, Claudio R. Mirasso, Fernanda S. Matias	Link	Neuronal heterogeneity, characterized by the presence of a multitude of spiking neuronal patterns, is a widespread phenomenon throughout the nervous system. In particular, the brain exhibits strong variability among inhibitory neurons. Despite the huge neuronal heterogeneity across brain regions, which in principle could decrease synchronization, cortical areas coherently oscillate during various cognitive tasks. Therefore, the functional significance of neuronal heterogeneity remains a subject of active investigation. Previous studies typically focus on the role of heterogeneity in the dynamic properties of only one population. Here, we explore how different types of inhibitory neurons can contribute to the diversity of the phase relations between two cortical areas. This research sheds light on the potential impact of local properties, such as neuronal variability, on communication between distant brain regions. We show that both homogeneous and heterogeneous inhibitory networks can exhibit phase diversity and nonintuitive regimes such as anticipated synchronization (AS) and phase bistability. It has been proposed that the bi-stable phase could be related to bi-stable perception, such as in the Necker cube. Moreover, we show that heterogeneity enlarges the region of zero-lag synchronization and bistability. We also show that the parameter controlling inhibitory heterogeneity modulates the transition from the usual delayed synchronization regime (DS) to AS. Finally, we show that the inhibitory heterogeneity drives the internal dynamics of the free-running population. Therefore, we suggest a possible mechanism to explain when the DS-AS transition occurs via zero-lag synchronization or bi-stability.
2024-11-29	A Comprehensive Framework for Automated Segmentation of Perivascular Spaces in Brain MRI with the nnU-Net	William Pham, Alexander Jarema, Donggyu Rim, Zhibin Chen, Mohamed S. H. Khlif, Vaughan G. Macefield, Luke A. Henderson, Amy Brodtmann	Link	Background: Enlargement of perivascular spaces (PVS) is common in neurodegenerative disorders including cerebral small vessel disease, Alzheimer's disease, and Parkinson's disease. PVS enlargement may indicate impaired clearance pathways and there is a need for reliable PVS detection methods which are currently lacking. Aim: To optimise a widely used deep learning model, the no-new-UNet (nnU-Net), for PVS segmentation. Methods: In 30 healthy participants (mean$\pm$SD age: 50$\pm$18.9 years; 13 females), T1-weighted MRI images were acquired using three different protocols on three MRI scanners (3T Siemens Tim Trio, 3T Philips Achieva, and 7T Siemens Magnetom). PVS were manually segmented across ten axial slices in each participant. Segmentations were completed using a sparse annotation strategy. In total, 11 models were compared using various strategies for image handling, preprocessing and semi-supervised learning with pseudo-labels. Model performance was evaluated using 5-fold cross validation (5FCV). The main performance metric was the Dice Similarity Coefficient (DSC). Results: The voxel-spacing agnostic model (mean$\pm$SD DSC=64.3$\pm$3.3%) outperformed models which resampled images to a common resolution (DSC=40.5-55%). Model performance improved substantially following iterative label cleaning (DSC=85.7$\pm$1.2%). Semi-supervised learning with pseudo-labels (n=12,740) from 18 additional datasets improved the agreement between raw and predicted PVS cluster counts (Lin's concordance correlation coefficient=0.89, 95%CI=0.82-0.94). We extended the model to enable PVS segmentation in the midbrain (DSC=64.3$\pm$6.5%) and hippocampus (DSC=67.8$\pm$5%). Conclusions: Our deep learning models provide a robust and holistic framework for the automated quantification of PVS in brain MRI.
2024-11-29	Protecting Multiple Types of Privacy Simultaneously in EEG-based Brain-Computer Interfaces	Lubin Meng, Xue Jiang, Tianwang Jia, Dongrui Wu	Link	A brain-computer interface (BCI) enables direct communication between the brain and an external device. Electroencephalogram (EEG) is the preferred input signal in non-invasive BCIs, due to its convenience and low cost. EEG-based BCIs have been successfully used in many applications, such as neurological rehabilitation, text input, games, and so on. However, EEG signals inherently carry rich personal information, necessitating privacy protection. This paper demonstrates that multiple types of private information (user identity, gender, and BCI-experience) can be easily inferred from EEG data, imposing a serious privacy threat to BCIs. To address this issue, we design perturbations to convert the original EEG data into privacy-protected EEG data, which conceal the private information while maintaining the primary BCI task performance. Experimental results demonstrated that the privacy-protected EEG data can significantly reduce the classification accuracy of user identity, gender and BCI-experience, but almost do not affect at all the classification accuracy of the primary BCI task, enabling user privacy protection in EEG-based BCIs.
2024-11-29	Actions and Objects Pathways for Domain Adaptation in Video Question Answering	Safaa Abdullahi Moallim Mohamud, Ho-Young Jung	Link	In this paper, we introduce the Actions and Objects Pathways (AOPath) for out-of-domain generalization in video question answering tasks. AOPath leverages features from a large pretrained model to enhance generalizability without the need for explicit training on the unseen domains. Inspired by human brain, AOPath dissociates the pretrained features into action and object features, and subsequently processes them through separate reasoning pathways. It utilizes a novel module which converts out-of-domain features into domain-agnostic features without introducing any trainable weights. We validate the proposed approach on the TVQA dataset, which is partitioned into multiple subsets based on genre to facilitate the assessment of generalizability. The proposed approach demonstrates 5% and 4% superior performance over conventional classifiers on out-of-domain and in-domain datasets, respectively. It also outperforms prior methods that involve training millions of parameters, whereas the proposed approach trains very few parameters.
2024-11-29	Core Placement Optimization of Many-core Brain-Inspired Near-Storage Systems for Spiking Neural Network Training	Xueke Zhu, Wenjie Lin, Yanyu Lin, Wenxiang Cheng, Zhengyu Ma, Yonghong Tian, Huihui Zhou	Link	With the increasing application scope of spiking neural networks (SNN), the complexity of SNN models has surged, leading to an exponential growth in demand for AI computility. As the new generation computing architecture of the neural networks, the efficiency and power consumption of distributed storage and parallel computing in the many-core near-memory computing system have attracted much attention. Among them, the mapping problem from logical cores to physical cores is one of the research hotspots. In order to improve the computing parallelism and system throughput of the many-core near-memory computing system, and to reduce power consumption, we propose a SNN training many-core deployment optimization method based on Off-policy Deterministic Actor-Critic. We utilize deep reinforcement learning as a nonlinear optimizer, treating the many-core topology as network graph features and using graph convolution to input the many-core structure into the policy network. We update the parameters of the policy network through near-end policy optimization to achieve deployment optimization of SNN models in the many-core near-memory computing architecture to reduce chip power consumption. To handle large-dimensional action spaces, we use continuous values matching the number of cores as the output of the policy network and then discretize them again to obtain new deployment schemes. Furthermore, to further balance inter-core computation latency and improve system throughput, we propose a model partitioning method with a balanced storage and computation strategy. Our method overcomes the problems such as uneven computation and storage loads between cores, and the formation of local communication hotspots, significantly reducing model training time, communication costs, and average flow load between cores in the many-core near-memory computing architecture.
2024-11-28	Fused-MemBrain: a spiking processor combining CMOS and self-assembled memristive networks	Davide Cipollini, Hugh Greatorex, Michele Mastella, Elisabetta Chicca, Lambert Schomaker	Link	In an era characterized by the rapid growth of data processing, developing new and efficient data processing technologies has become a priority. We address this by proposing a novel type of neuromorphic technology we call Fused-MemBrain. Our proposal is inspired by Golgi's theory modeling the brain as a syncytial continuum, in contrast to Cajal's theory of neurons and synapses being discrete elements. While Cajal's theory has long been the dominant and experimentally validated view of the nervous system, recent discoveries showed that a species of marine invertebrate (ctenophore Mnemiopsis leidyi) may be better described by Golgi's theory. The core idea is to develop hardware that functions analogously to a syncytial network, exploiting self-assembled memristive systems and combining them with CMOS technologies, interfacing with the silicon back-end-of-line. In this way, a memristive self-assembled material can cheaply and efficiently replace the synaptic connections between CMOS neuron implementations in neuromorphic hardware, enhancing the capability of massively parallel computation. The fusion of CMOS circuits with a memristive ``plexus'' allows information transfer without requiring engineered synapses, which typically consume significant area. As the first step toward this ambitious goal, we present a simulation of a memristive network interfaced with spiking neural networks. Additionally, we describe the potential benefits of such a system, along with key technical aspects it should incorporate.
2024-11-28	3D Wasserstein generative adversarial network with dense U-Net based discriminator for preclinical fMRI denoising	Sima Soltanpour, Arnold Chang, Dan Madularu, Praveen Kulkarni, Craig Ferris, Chris Joslin	Link	Functional magnetic resonance imaging (fMRI) is extensively used in clinical and preclinical settings to study brain function, however, fMRI data is inherently noisy due to physiological processes, hardware, and external noise. Denoising is one of the main preprocessing steps in any fMRI analysis pipeline. This process is challenging in preclinical data in comparison to clinical data due to variations in brain geometry, image resolution, and low signal-to-noise ratios. In this paper, we propose a structure-preserved algorithm based on a 3D Wasserstein generative adversarial network with a 3D dense U-net based discriminator called, 3D U-WGAN. We apply a 4D data configuration to effectively denoise temporal and spatial information in analyzing preclinical fMRI data. GAN-based denoising methods often utilize a discriminator to identify significant differences between denoised and noise-free images, focusing on global or local features. To refine the fMRI denoising model, our method employs a 3D dense U-Net discriminator to learn both global and local distinctions. To tackle potential over-smoothing, we introduce an adversarial loss and enhance perceptual similarity by measuring feature space distances. Experiments illustrate that 3D U-WGAN significantly improves image quality in resting-state and task preclinical fMRI data, enhancing signal-to-noise ratio without introducing excessive structural changes in existing methods. The proposed method outperforms state-of-the-art methods when applied to simulated and real data in a fMRI analysis pipeline.
2024-11-28	Sparse optimization for estimating the cross-power spectrum in linear inverse models : from theory to the application in brain connectivity	Laura Carini, Isabella Furci, Sara Sommariva	Link	In this work we present a computationally efficient linear optimization approach for estimating the cross--power spectrum of an hidden multivariate stochastic process from that of another observed process. Sparsity in the resulting estimator of the cross--power is induced through $\ell_1$ regularization and the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) is used for computing such an estimator. With respect to a standard implementation, we prove that a proper initialization step is sufficient to guarantee the required symmetric and antisymmetric properties of the involved quantities. Further, we show how structural properties of the forward operator can be exploited within the FISTA update in order to make our approach adequate also for large--scale problems such as those arising in context of brain functional connectivity. The effectiveness of the proposed approach is shown in a practical scenario where we aim at quantifying the statistical relationships between brain regions in the context of non-invasive electromagnetic field recordings. Our results show that our method provide results with an higher specificity that classical approaches based on a two--step procedure where first the hidden process describing the brain activity is estimated through a linear optimization step and then the cortical cross--power spectrum is computed from the estimated time--series.

EEG

Publish Date	Title	Authors	URL	Abstract
2024-11-29	Dynamic EEG-fMRI mapping: Revealing the relationship between brain connectivity and cognitive state	Guiran Liu, Binrong Zhu	Link	This study investigated the dynamic connectivity patterns between EEG and fMRI modalities, contributing to our understanding of brain network interactions. By employing a comprehensive approach that integrated static and dynamic analyses of EEG-fMRI data, we were able to uncover distinct connectivity states and characterize their temporal fluctuations. The results revealed modular organization within the intrinsic connectivity networks (ICNs) of the brain, highlighting the significant roles of sensory systems and the default mode network. The use of a sliding window technique allowed us to assess how functional connectivity varies over time, further elucidating the transient nature of brain connectivity. Additionally, our findings align with previous literature, reinforcing the notion that cognitive states can be effectively identified through short-duration data, specifically within the 30-60 second timeframe. The established relationships between connectivity strength and cognitive processes, particularly during different visual states, underscore the relevance of our approach for future research into brain dynamics. Overall, this study not only enhances our understanding of the interplay between EEG and fMRI signals but also paves the way for further exploration into the neural correlates of cognitive functions and their implications in clinical settings. Future research should focus on refining these methodologies and exploring their applications in various cognitive and clinical contexts.
2024-11-29	Graph-Enhanced EEG Foundation Model	Limin Wang, Toyotaro Suzumura, Hiroki Kanezashi	Link	Electroencephalography (EEG) signals provide critical insights for applications in disease diagnosis and healthcare. However, the scarcity of labeled EEG data poses a significant challenge. Foundation models offer a promising solution by leveraging large-scale unlabeled data through pre-training, enabling strong performance across diverse tasks. While both temporal dynamics and inter-channel relationships are vital for understanding EEG signals, existing EEG foundation models primarily focus on the former, overlooking the latter. To address this limitation, we propose a novel foundation model for EEG that integrates both temporal and inter-channel information. Our architecture combines Graph Neural Networks (GNNs), which effectively capture relational structures, with a masked autoencoder to enable efficient pre-training. We evaluated our approach using three downstream tasks and experimented with various GNN architectures. The results demonstrate that our proposed model, particularly when employing the GCN architecture with optimized configurations, consistently outperformed baseline methods across all tasks. These findings suggest that our model serves as a robust foundation model for EEG analysis.
2024-11-29	Knowledge-Data Fusion Based Source-Free Semi-Supervised Domain Adaptation for Seizure Subtype Classification	Ruimin Peng, Jiayu An, Dongrui Wu	Link	Electroencephalogram (EEG)-based seizure subtype classification enhances clinical diagnosis efficiency. Source-free semi-supervised domain adaptation (SF-SSDA), which transfers a pre-trained model to a new dataset with no source data and limited labeled target data, can be used for privacy-preserving seizure subtype classification. This paper considers two challenges in SF-SSDA for EEG-based seizure subtype classification: 1) How to effectively fuse both raw EEG data and expert knowledge in classifier design? 2) How to align the source and target domain distributions for SF-SSDA? We propose a Knowledge-Data Fusion based SF-SSDA approach, KDF-MutualSHOT, for EEG-based seizure subtype classification. In source model training, KDF uses Jensen-Shannon Divergence to facilitate mutual learning between a feature-driven Decision Tree-based model and a data-driven Transformer-based model. To adapt KDF to a new target dataset, an SF-SSDA algorithm, MutualSHOT, is developed, which features a consistency-based pseudo-label selection strategy. Experiments on the public TUSZ and CHSZ datasets demonstrated that KDF-MutualSHOT outperformed other supervised and source-free domain adaptation approaches in cross-subject seizure subtype classification.
2024-11-29	Protecting Multiple Types of Privacy Simultaneously in EEG-based Brain-Computer Interfaces	Lubin Meng, Xue Jiang, Tianwang Jia, Dongrui Wu	Link	A brain-computer interface (BCI) enables direct communication between the brain and an external device. Electroencephalogram (EEG) is the preferred input signal in non-invasive BCIs, due to its convenience and low cost. EEG-based BCIs have been successfully used in many applications, such as neurological rehabilitation, text input, games, and so on. However, EEG signals inherently carry rich personal information, necessitating privacy protection. This paper demonstrates that multiple types of private information (user identity, gender, and BCI-experience) can be easily inferred from EEG data, imposing a serious privacy threat to BCIs. To address this issue, we design perturbations to convert the original EEG data into privacy-protected EEG data, which conceal the private information while maintaining the primary BCI task performance. Experimental results demonstrated that the privacy-protected EEG data can significantly reduce the classification accuracy of user identity, gender and BCI-experience, but almost do not affect at all the classification accuracy of the primary BCI task, enabling user privacy protection in EEG-based BCIs.
2024-11-28	Pre-Training Graph Contrastive Masked Autoencoders are Strong Distillers for EEG	Xinxu Wei, Kanhao Zhao, Yong Jiao, Nancy B. Carlisle, Hua Xie, Yu Zhang	Link	Effectively utilizing extensive unlabeled high-density EEG data to improve performance in scenarios with limited labeled low-density EEG data presents a significant challenge. In this paper, we address this by framing it as a graph transfer learning and knowledge distillation problem. We propose a Unified Pre-trained Graph Contrastive Masked Autoencoder Distiller, named EEG-DisGCMAE, to bridge the gap between unlabeled/labeled and high/low-density EEG data. To fully leverage the abundant unlabeled EEG data, we introduce a novel unified graph self-supervised pre-training paradigm, which seamlessly integrates Graph Contrastive Pre-training and Graph Masked Autoencoder Pre-training. This approach synergistically combines contrastive and generative pre-training techniques by reconstructing contrastive samples and contrasting the reconstructions. For knowledge distillation from high-density to low-density EEG data, we propose a Graph Topology Distillation loss function, allowing a lightweight student model trained on low-density data to learn from a teacher model trained on high-density data, effectively handling missing electrodes through contrastive distillation. To integrate transfer learning and distillation, we jointly pre-train the teacher and student models by contrasting their queries and keys during pre-training, enabling robust distillers for downstream tasks. We demonstrate the effectiveness of our method on four classification tasks across two clinical EEG datasets with abundant unlabeled data and limited labeled data. The experimental results show that our approach significantly outperforms contemporary methods in both efficiency and accuracy.
2024-11-28	Decoding Imagined Movement in People with Multiple Sclerosis for Brain-Computer Interface Translation	John S. Russo, Thomas A. Shiels, Chin-Hsuan Sophie Lin, Sam E. John, David B. Grayden	Link	Multiple Sclerosis (MS) is a heterogeneous autoimmune-mediated disorder affecting the central nervous system, commonly manifesting as fatigue and progressive limb impairment. This can significantly impact quality of life due to weakness or paralysis in the upper and lower limbs. A Brain-Computer Interface (BCI) aims to restore quality of life through control of an external device, such as a wheelchair. However, the limited BCI research in people with MS is insufficient. The current study aims to expand on the current MS-BCI literature by highlighting the feasibility of decoding MS imagined movement. We collected electroencephalography (EEG) data from eight participants with various symptoms of MS and ten neurotypical control participants. Participants made imagined movements of the hands and feet as directed by a go no-go protocol. Binary regularised linear discriminant analysis was used to classify imagined movement at individual time-frequency points. The frequency bands which provided the maximal accuracy, and the associated latency, were compared. In all MS participants, the classification algorithm achieved above 70% accuracy in at least one imagined movement vs. rest classification and most movement vs. movement classifications. There was no significant difference between classification of limbs with weakness or paralysis to neurotypical controls. Both the MS and control groups possessed decodable information within the alpha (7-13 Hz) and beta (16-30 Hz) bands at similar latency. This study is the first to demonstrate the feasibility of decoding imagined movements in people with MS. As an alternative to the P300 response, motor imagery-based control of a BCI may also be combined with existing motor imagery therapy to supplement MS rehabilitation. These promising results merit further long term BCI studies to investigate the effect of MS progression on classification performance.
2024-11-28	ArEEG_Words: Dataset for Envisioned Speech Recognition using EEG for Arabic Words	Hazem Darwish, Abdalrahman Al Malah, Khloud Al Jallad, Nada Ghneim	Link	Brain-Computer-Interface (BCI) aims to support communication-impaired patients by translating neural signals into speech. A notable research topic in BCI involves Electroencephalography (EEG) signals that measure the electrical activity in the brain. While significant advancements have been made in BCI EEG research, a major limitation still exists: the scarcity of publicly available EEG datasets for non-English languages, such as Arabic. To address this gap, we introduce in this paper ArEEG_Words dataset, a novel EEG dataset recorded from 22 participants with mean age of 22 years (5 female, 17 male) using a 14-channel Emotiv Epoc X device. The participants were asked to be free from any effects on their nervous system, such as coffee, alcohol, cigarettes, and so 8 hours before recording. They were asked to stay calm in a clam room during imagining one of the 16 Arabic Words for 10 seconds. The words include 16 commonly used words such as up, down, left, and right. A total of 352 EEG recordings were collected, then each recording was divided into multiple 250ms signals, resulting in a total of 15,360 EEG signals. To the best of our knowledge, ArEEG_Words data is the first of its kind in Arabic EEG domain. Moreover, it is publicly available for researchers as we hope that will fill the gap in Arabic EEG research.
2024-11-27	EEG-Based Analysis of Brain Responses in Multi-Modal Human-Robot Interaction: Modulating Engagement	Suzanne Oliver, Tomoko Kitago, Adam Buchwald, S. Farokh Atashzar	Link	User engagement, cognitive participation, and motivation during task execution in physical human-robot interaction are crucial for motor learning. These factors are especially important in contexts like robotic rehabilitation, where neuroplasticity is targeted. However, traditional robotic rehabilitation systems often face challenges in maintaining user engagement, leading to unpredictable therapeutic outcomes. To address this issue, various techniques, such as assist-as-needed controllers, have been developed to prevent user slacking and encourage active participation. In this paper, we introduce a new direction through a novel multi-modal robotic interaction designed to enhance user engagement by synergistically integrating visual, motor, cognitive, and auditory (speech recognition) tasks into a single, comprehensive activity. To assess engagement quantitatively, we compared multiple electroencephalography (EEG) biomarkers between this multi-modal protocol and a traditional motor-only protocol. Fifteen healthy adult participants completed 100 trials of each task type. Our findings revealed that EEG biomarkers, particularly relative alpha power, showed statistically significant improvements in engagement during the multi-modal task compared to the motor-only task. Moreover, while engagement decreased over time in the motor-only task, the multi-modal protocol maintained consistent engagement, suggesting that users could remain engaged for longer therapy sessions. Our observations on neural responses during interaction indicate that the proposed multi-modal approach can effectively enhance user engagement, which is critical for improving outcomes. This is the first time that objective neural response highlights the benefit of a comprehensive robotic intervention combining motor, cognitive, and auditory functions in healthy subjects.
2024-11-27	The more, the better? Evaluating the role of EEG preprocessing for deep learning applications	Federico Del Pup, Andrea Zanola, Louis Fabrice Tshimanga, Alessandra Bertoldo, Manfredo Atzori	Link	The last decade has witnessed a notable surge in deep learning applications for the analysis of electroencephalography (EEG) data, thanks to its demonstrated superiority over conventional statistical techniques. However, even deep learning models can underperform if trained with bad processed data. While preprocessing is essential to the analysis of EEG data, there is a need of research examining its precise impact on model performance. This causes uncertainty about whether and to what extent EEG data should be preprocessed in a deep learning scenario. This study aims at investigating the role of EEG preprocessing in deep learning applications, drafting guidelines for future research. It evaluates the impact of different levels of preprocessing, from raw and minimally filtered data to complex pipelines with automated artifact removal algorithms. Six classification tasks (eye blinking, motor imagery, Parkinson's and Alzheimer's disease, sleep deprivation, and first episode psychosis) and four different architectures commonly used in the EEG domain were considered for the evaluation. The analysis of 4800 different trainings revealed statistical differences between the preprocessing pipelines at the intra-task level, for each of the investigated models, and at the inter-task level, for the largest one. Raw data generally leads to underperforming models, always ranking last in averaged score. In addition, models seem to benefit more from minimal pipelines without artifact handling methods, suggesting that EEG artifacts may contribute to the performance of deep neural networks.
2024-11-26	Evolving Markov Chains: Unsupervised Mode Discovery and Recognition from Data Streams	Kutalmış Coşkun, Borahan Tümer, Bjarne C. Hiller, Martin Becker	Link	Markov chains are simple yet powerful mathematical structures to model temporally dependent processes. They generally assume stationary data, i.e., fixed transition probabilities between observations/states. However, live, real-world processes, like in the context of activity tracking, biological time series, or industrial monitoring, often switch behavior over time. Such behavior switches can be modeled as transitions between higher-level \emph{modes} (e.g., running, walking, etc.). Yet all modes are usually not previously known, often exhibit vastly differing transition probabilities, and can switch unpredictably. Thus, to track behavior changes of live, real-world processes, this study proposes an online and efficient method to construct Evolving Markov chains (EMCs). EMCs adaptively track transition probabilities, automatically discover modes, and detect mode switches in an online manner. In contrast to previous work, EMCs are of arbitrary order, the proposed update scheme does not rely on tracking windows, only updates the relevant region of the probability tensor, and enjoys geometric convergence of the expected estimates. Our evaluation of synthetic data and real-world applications on human activity recognition, electric motor condition monitoring, and eye-state recognition from electroencephalography (EEG) measurements illustrates the versatility of the approach and points to the potential of EMCs to efficiently track, model, and understand live, real-world processes.

BCI

Publish Date	Title	Authors	URL	Abstract
2024-11-29	Protecting Multiple Types of Privacy Simultaneously in EEG-based Brain-Computer Interfaces	Lubin Meng, Xue Jiang, Tianwang Jia, Dongrui Wu	Link	A brain-computer interface (BCI) enables direct communication between the brain and an external device. Electroencephalogram (EEG) is the preferred input signal in non-invasive BCIs, due to its convenience and low cost. EEG-based BCIs have been successfully used in many applications, such as neurological rehabilitation, text input, games, and so on. However, EEG signals inherently carry rich personal information, necessitating privacy protection. This paper demonstrates that multiple types of private information (user identity, gender, and BCI-experience) can be easily inferred from EEG data, imposing a serious privacy threat to BCIs. To address this issue, we design perturbations to convert the original EEG data into privacy-protected EEG data, which conceal the private information while maintaining the primary BCI task performance. Experimental results demonstrated that the privacy-protected EEG data can significantly reduce the classification accuracy of user identity, gender and BCI-experience, but almost do not affect at all the classification accuracy of the primary BCI task, enabling user privacy protection in EEG-based BCIs.
2024-11-28	Decoding Imagined Movement in People with Multiple Sclerosis for Brain-Computer Interface Translation	John S. Russo, Thomas A. Shiels, Chin-Hsuan Sophie Lin, Sam E. John, David B. Grayden	Link	Multiple Sclerosis (MS) is a heterogeneous autoimmune-mediated disorder affecting the central nervous system, commonly manifesting as fatigue and progressive limb impairment. This can significantly impact quality of life due to weakness or paralysis in the upper and lower limbs. A Brain-Computer Interface (BCI) aims to restore quality of life through control of an external device, such as a wheelchair. However, the limited BCI research in people with MS is insufficient. The current study aims to expand on the current MS-BCI literature by highlighting the feasibility of decoding MS imagined movement. We collected electroencephalography (EEG) data from eight participants with various symptoms of MS and ten neurotypical control participants. Participants made imagined movements of the hands and feet as directed by a go no-go protocol. Binary regularised linear discriminant analysis was used to classify imagined movement at individual time-frequency points. The frequency bands which provided the maximal accuracy, and the associated latency, were compared. In all MS participants, the classification algorithm achieved above 70% accuracy in at least one imagined movement vs. rest classification and most movement vs. movement classifications. There was no significant difference between classification of limbs with weakness or paralysis to neurotypical controls. Both the MS and control groups possessed decodable information within the alpha (7-13 Hz) and beta (16-30 Hz) bands at similar latency. This study is the first to demonstrate the feasibility of decoding imagined movements in people with MS. As an alternative to the P300 response, motor imagery-based control of a BCI may also be combined with existing motor imagery therapy to supplement MS rehabilitation. These promising results merit further long term BCI studies to investigate the effect of MS progression on classification performance.
2024-11-28	ArEEG_Words: Dataset for Envisioned Speech Recognition using EEG for Arabic Words	Hazem Darwish, Abdalrahman Al Malah, Khloud Al Jallad, Nada Ghneim	Link	Brain-Computer-Interface (BCI) aims to support communication-impaired patients by translating neural signals into speech. A notable research topic in BCI involves Electroencephalography (EEG) signals that measure the electrical activity in the brain. While significant advancements have been made in BCI EEG research, a major limitation still exists: the scarcity of publicly available EEG datasets for non-English languages, such as Arabic. To address this gap, we introduce in this paper ArEEG_Words dataset, a novel EEG dataset recorded from 22 participants with mean age of 22 years (5 female, 17 male) using a 14-channel Emotiv Epoc X device. The participants were asked to be free from any effects on their nervous system, such as coffee, alcohol, cigarettes, and so 8 hours before recording. They were asked to stay calm in a clam room during imagining one of the 16 Arabic Words for 10 seconds. The words include 16 commonly used words such as up, down, left, and right. A total of 352 EEG recordings were collected, then each recording was divided into multiple 250ms signals, resulting in a total of 15,360 EEG signals. To the best of our knowledge, ArEEG_Words data is the first of its kind in Arabic EEG domain. Moreover, it is publicly available for researchers as we hope that will fill the gap in Arabic EEG research.
2024-11-25	Explainable MST-ECoGNet Decode Visual Information from ECoG Signal	Changqing JI	Link	In the application of brain-computer interface (BCI), we not only need to accurately decode brain signals,but also need to consider the explainability of the decoding process, which is related to the reliability of the model. In the process of designing a decoder or processing brain signals, we need to explain the discovered phenomena in physical or physiological way. An explainable model not only makes the signal processing process clearer and improves reliability, but also allows us to better understand brain activities and facilitate further exploration of the brain. In this paper, we systematically analyze the multi-classification dataset of visual brain signals ECoG, using a simple and highly explainable method to explore the ways in which ECoG carry visual information, then based on these findings, we propose a model called MST-ECoGNet that combines traditional mathematics and deep learning. The main contributions of this paper are: 1) found that ECoG time-frequency domain information carries visual information, provides important features for visual classification tasks. The mathematical method of MST (Modified S Transform) can effectively extract temporal-frequency domain information; 2) The spatial domain of ECoG signals also carries visual information, the unique spatial features are also important features for classification tasks; 3) The real and imaginary information in the time-frequency domain are complementary. The effective combination of the two is more helpful for classification tasks than using amplitude information alone; 4) Finally, compared with previous work, our model is smaller and has higher performance: for the object MonJ, the model size is reduced to 10.82% of base model, the accuracy is improved by 6.63%; for the object MonC, the model size is reduced to 8.78%, the accuracy is improved by 16.63%.
2024-11-23	ChatBCI: A P300 Speller BCI Leveraging Large Language Models for Improved Sentence Composition in Realistic Scenarios	Jiazhen Hong, Weinan Wang, Laleh Najafizadeh	Link	P300 speller BCIs allow users to compose sentences by selecting target keys on a GUI through the detection of P300 component in their EEG signals following visual stimuli. Most P300 speller BCIs require users to spell words letter by letter, or the first few initial letters, resulting in high keystroke demands that increase time, cognitive load, and fatigue. This highlights the need for more efficient, user-friendly methods for faster sentence composition. In this work, we introduce ChatBCI, a P300 speller BCI that leverages the zero-shot learning capabilities of large language models (LLMs) to suggest words from user-spelled initial letters or predict the subsequent word(s), reducing keystrokes and accelerating sentence composition. ChatBCI retrieves word suggestions through remote queries to the GPT-3.5 API. A new GUI, displaying GPT-3.5 word suggestions as extra keys is designed. SWLDA is used for the P300 classification. Seven subjects completed two online spelling tasks: 1) copy-spelling a self-composed sentence using ChatBCI, and 2) improvising a sentence using ChatBCI's word suggestions. Results demonstrate that in Task 1, on average, ChatBCI outperforms letter-by-letter BCI spellers, reducing time and keystrokes by 62.14% and 53.22%, respectively, and increasing information transfer rate by 198.96%. In Task 2, ChatBCI achieves 80.68% keystroke savings and a record 8.53 characters/min for typing speed. Overall, ChatBCI, by employing remote LLM queries, enhances sentence composition in realistic scenarios, significantly outperforming traditional spellers without requiring local model training or storage. ChatBCI's (multi-) word predictions, combined with its new GUI, pave the way for developing next-generation speller BCIs that are efficient and effective for real-time communication, especially for users with communication and motor disabilities.
2024-11-22	Brain-Computer Interfaces for Emotional Regulation in Patients with Various Disorders	Vedant Mehta	Link	Neurological and Physiological Disorders that impact emotional regulation each have their own unique characteristics which are important to understand in order to create a generalized solution to all of them. The purpose of this experiment is to explore the potential applications of EEG-based Brain-Computer Interfaces (BCIs) in enhancing emotional regulation for individuals with neurological and physiological disorders. The research focuses on the development of a novel neural network algorithm for understanding EEG data, with a particular emphasis on recognizing and regulating emotional states. The procedure involves the collection of EEG-based emotion data from open-Neuro. Using novel data modification techniques, information from the dataset can be altered to create a dataset that has neural patterns of patients with disorders whilst showing emotional change. The data analysis reveals promising results, as the algorithm is able to successfully classify emotional states with a high degree of accuracy. This suggests that EEG-based BCIs have the potential to be a valuable tool in aiding individuals with a range of neurological and physiological disorders in recognizing and regulating their emotions. To improve upon this work, data collection on patients with neurological disorders should be done to improve overall sample diversity.
2024-11-19	Enhanced Cross-Dataset Electroencephalogram-based Emotion Recognition using Unsupervised Domain Adaptation	Md Niaz Imtiaz, Naimul Khan	Link	Emotion recognition has significant potential in healthcare and affect-sensitive systems such as brain-computer interfaces (BCIs). However, challenges such as the high cost of labeled data and variability in electroencephalogram (EEG) signals across individuals limit the applicability of EEG-based emotion recognition models across domains. These challenges are exacerbated in cross-dataset scenarios due to differences in subject demographics, recording devices, and presented stimuli. To address these issues, we propose a novel approach to improve cross-domain EEG-based emotion classification. Our method, Gradual Proximity-guided Target Data Selection (GPTDS), incrementally selects reliable target domain samples for training. By evaluating their proximity to source clusters and the models confidence in predicting them, GPTDS minimizes negative transfer caused by noisy and diverse samples. Additionally, we introduce Prediction Confidence-aware Test-Time Augmentation (PC-TTA), a cost-effective augmentation technique. Unlike traditional TTA methods, which are computationally intensive, PC-TTA activates only when model confidence is low, improving inference performance while drastically reducing computational costs. Experiments on the DEAP and SEED datasets validate the effectiveness of our approach. When trained on DEAP and tested on SEED, our model achieves 67.44% accuracy, a 7.09% improvement over the baseline. Conversely, training on SEED and testing on DEAP yields 59.68% accuracy, a 6.07% improvement. Furthermore, PC-TTA reduces computational time by a factor of 15 compared to traditional TTA methods. Our method excels in detecting both positive and negative emotions, demonstrating its practical utility in healthcare applications. Code available at: https://github.com/RyersonMultimediaLab/EmotionRecognitionUDA
2024-11-18	Towards Personalized Brain-Computer Interface Application Based on Endogenous EEG Paradigms	Heon-Gyu Kwak, Gi-Hwan Shin, Yeon-Woo Choi, Dong-Hoon Lee, Yoo-In Jeon, Jun-Su Kang, Seong-Whan Lee	Link	In this paper, we propose a conceptual framework for personalized brain-computer interface (BCI) applications, which can offer an enhanced user experience by customizing services to individual preferences and needs, based on endogenous electroencephalography (EEG) paradigms including motor imagery (MI), speech imagery (SI), and visual imagery. The framework includes two essential components: user identification and intention classification, which enable personalized services by identifying individual users and recognizing their intended actions through EEG signals. We validate the feasibility of our framework using a private EEG dataset collected from eight subjects, employing the ShallowConvNet architecture to decode EEG features. The experimental results demonstrate that user identification achieved an average classification accuracy of 0.995, while intention classification achieved 0.47 accuracy across all paradigms, with MI demonstrating the best performance. These findings indicate that EEG signals can effectively support personalized BCI applications, offering robust identification and reliable intention decoding, especially for MI and SI.
2024-11-29	Imagined Speech and Visual Imagery as Intuitive Paradigms for Brain-Computer Interfaces	Seo-Hyun Lee, Ji-Ha Park, Deok-Seon Kim	Link	Brain-computer interfaces (BCIs) have shown promise in enabling communication for individuals with motor impairments. Recent advancements like brain-to-speech technology aim to reconstruct speech from neural activity. However, decoding communication-related paradigms, such as imagined speech and visual imagery, using non-invasive techniques remains challenging. This study analyzes brain dynamics in these two paradigms by examining neural synchronization and functional connectivity through phase-locking values (PLV) in EEG data from 16 participants. Results show that visual imagery produces higher PLV values in visual cortex, engaging spatial networks, while imagined speech demonstrates consistent synchronization, primarily engaging language-related regions. These findings suggest that imagined speech is suitable for language-driven BCI applications, while visual imagery can complement BCI systems for users with speech impairments. Personalized calibration is crucial for optimizing BCI performance.
2024-11-12	EEG-DCNet: A Fast and Accurate MI-EEG Dilated CNN Classification Method	Wei Peng, Kang Liu, Jiaxi Shi, Jianchen Hu	Link	The electroencephalography (EEG)-based motor imagery (MI) classification is a critical and challenging task in brain-computer interface (BCI) technology, which plays a significant role in assisting patients with functional impairments to regain mobility. We present a novel multi-scale atrous convolutional neural network (CNN) model called EEG-dilated convolution network (DCNet) to enhance the accuracy and efficiency of the EEG-based MI classification tasks. We incorporate the $1\times1$ convolutional layer and utilize the multi-branch parallel atrous convolutional architecture in EEG-DCNet to capture the highly nonlinear characteristics and multi-scale features of the EEG signals. Moreover, we utilize the sliding window to enhance the temporal consistency and utilize the attension mechanism to improve the accuracy of recognizing user intentions. The experimental results (via the BCI-IV-2a ,BCI-IV-2b and the High-Gamma datasets) show that EEG-DCNet outperforms existing state-of-the-art (SOTA) approaches in terms of classification accuracy and Kappa scores. Furthermore, since EEG-DCNet requires less number of parameters, the training efficiency and memory consumption are also improved. The experiment code is open-sourced at \href{https://github.com/Kanyooo/EEG-DCNet}{here}.

fMRI

Publish Date	Title	Authors	URL	Abstract
2024-11-29	Dynamic EEG-fMRI mapping: Revealing the relationship between brain connectivity and cognitive state	Guiran Liu, Binrong Zhu	Link	This study investigated the dynamic connectivity patterns between EEG and fMRI modalities, contributing to our understanding of brain network interactions. By employing a comprehensive approach that integrated static and dynamic analyses of EEG-fMRI data, we were able to uncover distinct connectivity states and characterize their temporal fluctuations. The results revealed modular organization within the intrinsic connectivity networks (ICNs) of the brain, highlighting the significant roles of sensory systems and the default mode network. The use of a sliding window technique allowed us to assess how functional connectivity varies over time, further elucidating the transient nature of brain connectivity. Additionally, our findings align with previous literature, reinforcing the notion that cognitive states can be effectively identified through short-duration data, specifically within the 30-60 second timeframe. The established relationships between connectivity strength and cognitive processes, particularly during different visual states, underscore the relevance of our approach for future research into brain dynamics. Overall, this study not only enhances our understanding of the interplay between EEG and fMRI signals but also paves the way for further exploration into the neural correlates of cognitive functions and their implications in clinical settings. Future research should focus on refining these methodologies and exploring their applications in various cognitive and clinical contexts.
2024-11-28	3D Wasserstein generative adversarial network with dense U-Net based discriminator for preclinical fMRI denoising	Sima Soltanpour, Arnold Chang, Dan Madularu, Praveen Kulkarni, Craig Ferris, Chris Joslin	Link	Functional magnetic resonance imaging (fMRI) is extensively used in clinical and preclinical settings to study brain function, however, fMRI data is inherently noisy due to physiological processes, hardware, and external noise. Denoising is one of the main preprocessing steps in any fMRI analysis pipeline. This process is challenging in preclinical data in comparison to clinical data due to variations in brain geometry, image resolution, and low signal-to-noise ratios. In this paper, we propose a structure-preserved algorithm based on a 3D Wasserstein generative adversarial network with a 3D dense U-net based discriminator called, 3D U-WGAN. We apply a 4D data configuration to effectively denoise temporal and spatial information in analyzing preclinical fMRI data. GAN-based denoising methods often utilize a discriminator to identify significant differences between denoised and noise-free images, focusing on global or local features. To refine the fMRI denoising model, our method employs a 3D dense U-Net discriminator to learn both global and local distinctions. To tackle potential over-smoothing, we introduce an adversarial loss and enhance perceptual similarity by measuring feature space distances. Experiments illustrate that 3D U-WGAN significantly improves image quality in resting-state and task preclinical fMRI data, enhancing signal-to-noise ratio without introducing excessive structural changes in existing methods. The proposed method outperforms state-of-the-art methods when applied to simulated and real data in a fMRI analysis pipeline.
2024-11-25	Energy landscape analysis based on the Ising model: Tutorial review	Naoki Masuda, Saiful Islam, Si Thu Aung, Takamitsu Watanabe	Link	We review a class of energy landscape analysis method that uses the Ising model and takes multivariate time series data as input. The method allows one to capture dynamics of the data as trajectories of a ball from one basin to a different basin to yet another, constrained on the energy landscape specified by the estimated Ising model. While this energy landscape analysis has mostly been applied to functional magnetic resonance imaging (fMRI) data from the brain for historical reasons, there are emerging applications outside fMRI data and neuroscience. To inform such applications in various research fields, this review paper provides a detailed tutorial on each step of the analysis, terminologies, concepts underlying the method, and validation, as well as recent developments of extended and related methods.
2024-11-25	COBRA: A Continual Learning Approach to Vision-Brain Understanding	Xuan-Bac Nguyen, Arabinda Kumar Choudhary, Pawan Sinha, Xin Li, Khoa Luu	Link	Vision-Brain Understanding (VBU) aims to extract visual information perceived by humans from brain activity recorded through functional Magnetic Resonance Imaging (fMRI). Despite notable advancements in recent years, existing studies in VBU continue to face the challenge of catastrophic forgetting, where models lose knowledge from prior subjects as they adapt to new ones. Addressing continual learning in this field is, therefore, essential. This paper introduces a novel framework called Continual Learning for Vision-Brain (COBRA) to address continual learning in VBU. Our approach includes three novel modules: a Subject Commonality (SC) module, a Prompt-based Subject Specific (PSS) module, and a transformer-based module for fMRI, denoted as MRIFormer module. The SC module captures shared vision-brain patterns across subjects, preserving this knowledge as the model encounters new subjects, thereby reducing the impact of catastrophic forgetting. On the other hand, the PSS module learns unique vision-brain patterns specific to each subject. Finally, the MRIFormer module contains a transformer encoder and decoder that learns the fMRI features for VBU from common and specific patterns. In a continual learning setup, COBRA is trained in new PSS and MRIFormer modules for new subjects, leaving the modules of previous subjects unaffected. As a result, COBRA effectively addresses catastrophic forgetting and achieves state-of-the-art performance in both continual learning and vision-brain reconstruction tasks, surpassing previous methods.
2024-11-25	Stability of Brain Functional Network During Working Memory Using Structural Balance Theory	Sepehr Gourabi, Masoud Lotfalipour, Reza Khosrowabadi, Reza Jafari	Link	Working memory plays a crucial role in various aspects of human life. Therefore, it has been an area of interest in different research studies, especially neuroscience. The neuroscientists investigating working memory have primarily emphasized the brain's functional modularity. At the same time, a holistic perspective is still required to investigate the brain as an integrated and unified system. We hypothesized that the brain should shift towards a more stable state during working memory than the resting state. Therefore, based on the Structural Balance Theory (SBT), we aimed to address this process. To achieve this, we examined triadic associations in signed fMRI networks in healthy individuals using the N-back as the working memory task. We demonstrated that the number of balanced triads increased during the working memory task compared to the resting state, while the opposite is true for imbalanced triads. The increase of balanced triads forced the network to a more stable state with a lower balance energy level. The increase of balanced triads was crucially related to changes in anti-synchrony to synchronous activities between the Temporal Cortex, the Prefrontal Cortex, and the Parietal Cortex, which are known to be involved in various aspects of working memory, during the working memory process. We hope these findings pave the way to a better understanding the working memory process.
2024-11-23	Inducing Human-like Biases in Moral Reasoning Language Models	Artem Karpov, Seong Hah Cho, Austin Meek, Raymond Koopmanschap, Lucy Farnik, Bogdan-Ionut Cirstea	Link	In this work, we study the alignment (BrainScore) of large language models (LLMs) fine-tuned for moral reasoning on behavioral data and/or brain data of humans performing the same task. We also explore if fine-tuning several LLMs on the fMRI data of humans performing moral reasoning can improve the BrainScore. We fine-tune several LLMs (BERT, RoBERTa, DeBERTa) on moral reasoning behavioral data from the ETHICS benchmark [Hendrycks et al., 2020], on the moral reasoning fMRI data from Koster-Hale et al. [2013], or on both. We study both the accuracy on the ETHICS benchmark and the BrainScores between model activations and fMRI data. While larger models generally performed better on both metrics, BrainScores did not significantly improve after fine-tuning.
2024-11-20	Quantum-Brain: Quantum-Inspired Neural Network Approach to Vision-Brain Understanding	Hoang-Quan Nguyen, Xuan-Bac Nguyen, Hugh Churchill, Arabinda Kumar Choudhary, Pawan Sinha, Samee U. Khan, Khoa Luu	Link	Vision-brain understanding aims to extract semantic information about brain signals from human perceptions. Existing deep learning methods for vision-brain understanding are usually introduced in a traditional learning paradigm missing the ability to learn the connectivities between brain regions. Meanwhile, the quantum computing theory offers a new paradigm for designing deep learning models. Motivated by the connectivities in the brain signals and the entanglement properties in quantum computing, we propose a novel Quantum-Brain approach, a quantum-inspired neural network, to tackle the vision-brain understanding problem. To compute the connectivity between areas in brain signals, we introduce a new Quantum-Inspired Voxel-Controlling module to learn the impact of a brain voxel on others represented in the Hilbert space. To effectively learn connectivity, a novel Phase-Shifting module is presented to calibrate the value of the brain signals. Finally, we introduce a new Measurement-like Projection module to present the connectivity information from the Hilbert space into the feature space. The proposed approach can learn to find the connectivities between fMRI voxels and enhance the semantic information obtained from human perceptions. Our experimental results on the Natural Scene Dataset benchmarks illustrate the effectiveness of the proposed method with Top-1 accuracies of 95.1% and 95.6% on image and brain retrieval tasks and an Inception score of 95.3% on fMRI-to-image reconstruction task. Our proposed quantum-inspired network brings a potential paradigm to solving the vision-brain problems via the quantum computing theory.
2024-11-19	Enhancing Blind Source Separation with Dissociative Principal Component Analysis	Muhammad Usman Khalid	Link	Sparse principal component analysis (sPCA) enhances the interpretability of principal components (PCs) by imposing sparsity constraints on loading vectors (LVs). However, when used as a precursor to independent component analysis (ICA) for blind source separation (BSS), sPCA may underperform due to its focus on simplicity, potentially disregarding some statistical information essential for effective ICA. To overcome this limitation, a sophisticated approach is proposed that preserves the interpretability advantages of sPCA while significantly enhancing its source extraction capabilities. This consists of two tailored algorithms, dissociative PCA (DPCA1 and DPCA2), which employ adaptive and firm thresholding alongside gradient and coordinate descent approaches to optimize the proposed model dynamically. These algorithms integrate left and right singular vectors from singular value decomposition (SVD) through dissociation matrices (DMs) that replace traditional singular values, thus capturing latent interdependencies effectively to model complex source relationships. This leads to refined PCs and LVs that more accurately represent the underlying data structure. The proposed approach avoids focusing on individual eigenvectors, instead, it collaboratively combines multiple eigenvectors to disentangle interdependencies within each SVD variate. The superior performance of the proposed DPCA algorithms is demonstrated across four varied imaging applications including functional magnetic resonance imaging (fMRI) source retrieval, foreground-background separation, image reconstruction, and image inpainting. They outperformed traditional methods such as PCA+ICA, PPCA+ICA, SPCA+ICA, PMD, and GPower.
2024-11-16	In silico discovery of representational relationships across visual cortex	Alessandro T. Gifford, Maya A. Jastrzębowska, Johannes J. D. Singer, Radoslaw M. Cichy	Link	Human vision is mediated by a complex interconnected network of cortical brain areas jointly representing visual information. While these areas are increasingly understood in isolation, their representational relationships remain elusive. Here we developed relational neural control (RNC), and used it to investigate the representational relationships for univariate and multivariate fMRI responses of early- and mid-level visual areas. RNC generated and explored in silico fMRI responses for large amounts of images, discovering controlling images that align or disentangle responses across areas, thus indicating their shared or unique representational content. A large portion of representational content was shared across areas, unique representational content increased with cortical distance, and we isolated the visual features determining these effects. Closing the empirical cycle, we validated the in silico discoveries on in vivo fMRI responses from independent subjects. Together, this reveals how visual areas jointly represent the world as an interconnected network.
2024-11-14	Towards Neural Foundation Models for Vision: Aligning EEG, MEG, and fMRI Representations for Decoding, Encoding, and Modality Conversion	Matteo Ferrante, Tommaso Boccato, Grigorii Rashkov, Nicola Toschi	Link	This paper presents a novel approach towards creating a foundational model for aligning neural data and visual stimuli across multimodal representationsof brain activity by leveraging contrastive learning. We used electroencephalography (EEG), magnetoencephalography (MEG), and functional magnetic resonance imaging (fMRI) data. Our framework's capabilities are demonstrated through three key experiments: decoding visual information from neural data, encoding images into neural representations, and converting between neural modalities. The results highlight the model's ability to accurately capture semantic information across different brain imaging techniques, illustrating its potential in decoding, encoding, and modality conversion tasks.

MEG

Publish Date	Title	Authors	URL	Abstract
2024-11-29	Neuroplasticity and Psychedelics: a comprehensive examination of classic and non-classic compounds in pre and clinical models	Claudio Agnorelli, Meg Spriggs, Kate Godfrey, Gabriela Sawicka, Bettina Bohl, Hannah Douglass, Andrea Fagiolini, Hashemi Parastoo, Robin Carhart-Harris, David Nutt, David Erritzoe	Link	Neuroplasticity, the ability of the nervous system to adapt throughout an organism's lifespan, offers potential as both a biomarker and treatment target for neuropsychiatric conditions. Psychedelics, a burgeoning category of drugs, are increasingly prominent in psychiatric research, prompting inquiries into their mechanisms of action. Distinguishing themselves from traditional medications, psychedelics demonstrate rapid and enduring therapeutic effects after a single or few administrations, believed to stem from their neuroplasticity-enhancing properties. This review examines how classic psychedelics (e.g., LSD, psilocybin, N,N-DMT) and non-classic psychedelics (e.g., ketamine, MDMA) influence neuroplasticity. Drawing from preclinical and clinical studies, we explore the molecular, structural, and functional changes triggered by these agents. Animal studies suggest psychedelics induce heightened sensitivity of the nervous system to environmental stimuli (meta-plasticity), re-opening developmental windows for long-term structural changes (hyper-plasticity), with implications for mood and behavior. Translating these findings to humans faces challenges due to limitations in current imaging techniques. Nonetheless, promising new directions for human research are emerging, including the employment of novel positron-emission tomography (PET) radioligands, non-invasive brain stimulation methods, and multimodal approaches. By elucidating the interplay between psychedelics and neuroplasticity, this review informs the development of targeted interventions for neuropsychiatric disorders and advances understanding of psychedelics' therapeutic potential.
2024-11-29	On Monitoring Edge-Geodetic Sets of Dynamic Graph	Zin Mar Myint, Ashish Saxena	Link	The concept of a monitoring edge-geodetic set (MEG-set) in a graph $G$, denoted $MEG(G)$, refers to a subset of vertices $MEG(G)\subseteq V(G)$ such that every edge $e$ in $G$ is monitored by some pair of vertices $ u, v \in MEG(G)$, where $e$ lies on all shortest paths between $u$ and $v$. The minimum number of vertices required to form such a set is called the monitoring edge-geodetic number, denoted $meg(G)$. The primary motivation for studying $MEG$-sets in previous works arises from scenarios in which certain edges are removed from $G$. In these cases, the vertices of the $MEG$-set are responsible for detecting these deletions. Such detection is crucial for identifying which edges have been removed from $G$ and need to be repaired. In real life, repairing these edges may be costly, or sometimes it is impossible to repair edges. In this case, the original $MEG$-set may no longer be effective in monitoring the modified graph. This highlights the importance of reassessing and adapting the $MEG$-set after edge deletions. This work investigates the monitoring edge-geodetic properties of graphs, focusing on how the removal of $k$ edges affects the structure of a graph and influences its monitoring capabilities. Specifically, we explore how the monitoring edge-geodetic number $meg(G)$ changes when $k$ edges are removed. The study aims to compare the monitoring properties of the original graph with those of the modified graph and to understand the impact of edge deletions.
2024-11-14	Towards Neural Foundation Models for Vision: Aligning EEG, MEG, and fMRI Representations for Decoding, Encoding, and Modality Conversion	Matteo Ferrante, Tommaso Boccato, Grigorii Rashkov, Nicola Toschi	Link	This paper presents a novel approach towards creating a foundational model for aligning neural data and visual stimuli across multimodal representationsof brain activity by leveraging contrastive learning. We used electroencephalography (EEG), magnetoencephalography (MEG), and functional magnetic resonance imaging (fMRI) data. Our framework's capabilities are demonstrated through three key experiments: decoding visual information from neural data, encoding images into neural representations, and converting between neural modalities. The results highlight the model's ability to accurately capture semantic information across different brain imaging techniques, illustrating its potential in decoding, encoding, and modality conversion tasks.
2024-11-12	Search for the X17 particle in $^{7}\mathrm{Li}(\mathrm{p},\mathrm{e}^+ \mathrm{e}^{-}) ^{8}\mathrm{Be}$ processes with the MEG II detector	The MEG II collaboration, K. Afanaciev, A. M. Baldini, S. Ban, H. Benmansour, G. Boca, P. W. Cattaneo, G. Cavoto, F. Cei, M. Chiappini, A. Corvaglia, G. Dal Maso, A. De Bari, M. De Gerone, L. Ferrari Barusso, M. Francesconi, L. Galli, G. Gallucci, F. Gatti, L. Gerritzen, F. Grancagnolo, E. G. Grandoni, M. Grassi, D. N. Grigoriev, M. Hildebrandt, F. Ignatov, F. Ikeda, T. Iwamoto, S. Karpov, P. -R. Kettle, N. Khomutov, A. Kolesnikov, N. Kravchuk, V. Krylov, N. Kuchinskiy, F. Leonetti, W. Li, V. Malyshev, A. Matsushita, M. Meucci, S. Mihara, W. Molzon, T. Mori, D. Nicolò, H. Nishiguchi, A. Ochi, W. Ootani, A. Oya, D. Palo, M. Panareo, A. Papa, V. Pettinacci, A. Popov, F. Renga, S. Ritt, M. Rossella, A. Rozhdestvensky. S. Scarpellini, P. Schwendimann, G. Signorelli, M. Takahashi, Y. Uchiyama, A. Venturini, B. Vitali, C. Voena, K. Yamamoto, R. Yokota, T. Yonemoto	Link	The observation of a resonance structure in the opening angle of the electron-positron pairs in the $^{7}$Li(p,\ee) $^{8}$Be reaction was claimed and interpreted as the production and subsequent decay of a hypothetical particle (X17). Similar excesses, consistent with this particle, were later observed in processes involving $^{4}$He and $^{12}$C nuclei with the same experimental technique. The MEG II apparatus at PSI, designed to search for the $\mu^+ \rightarrow \mathrm{e}^+ \gamma$ decay, can be exploited to investigate the existence of this particle and study its nature. Protons from a Cockroft-Walton accelerator, with an energy up to 1.1 MeV, were delivered on a dedicated Li-based target. The $\gamma$ and the e$^{+}$e$^{-}$ pair emerging from the $^8\mathrm{Be}^*$ transitions were studied with calorimeters and a spectrometer, featuring a broader angular acceptance than previous experiments. We present in this paper the analysis of a four-week data-taking in 2023 with a beam energy of 1080 keV, resulting in the excitation of two different resonances with Q-value \SI{17.6}{\mega\electronvolt} and \SI{18.1}{\mega\electronvolt}. No significant signal was found, and limits at \SI{90}{\percent} C.L. on the branching ratios (relative to the $\gamma$ emission) of the two resonances to X17 were set, $R_{17.6} < 1.8 \times 10^{-6} $ and $R_{18.1} &lt; 1.2 \times 10^{-5} $.
2024-11-07	MEG: Medical Knowledge-Augmented Large Language Models for Question Answering	Laura Cabello, Carmen Martin-Turrero, Uchenna Akujuobi, Anders Søgaard, Carlos Bobed	Link	Question answering is a natural language understanding task that involves reasoning over both explicit context and unstated, relevant domain knowledge. Large language models (LLMs), which underpin most contemporary question answering systems, struggle to induce how concepts relate in specialized domains such as medicine. Existing medical LLMs are also costly to train. In this work, we present MEG, a parameter-efficient approach for medical knowledge-augmented LLMs. MEG uses a lightweight mapping network to integrate graph embeddings into the LLM, enabling it to leverage external knowledge in a cost-effective way. We evaluate our method on four popular medical multiple-choice datasets and show that LLMs greatly benefit from the factual grounding provided by knowledge graph embeddings. MEG attains an average of +10.2% accuracy over the Mistral-Instruct baseline, and +6.7% over specialized models like BioMistral. We also show results based on Llama-3. Finally, we show that MEG's performance remains robust to the choice of graph encoder.
2024-10-30	STIED: A deep learning model for the SpatioTemporal detection of focal Interictal Epileptiform Discharges with MEG	Raquel Fernández-Martín, Alfonso Gijón, Odile Feys, Elodie Juvené, Alec Aeby, Charline Urbain, Xavier De Tiège, Vincent Wens	Link	Magnetoencephalography (MEG) allows the non-invasive detection of interictal epileptiform discharges (IEDs). Clinical MEG analysis in epileptic patients traditionally relies on the visual identification of IEDs, which is time consuming and partially subjective. Automatic, data-driven detection methods exist but show limited performance. Still, the rise of deep learning (DL)-with its ability to reproduce human-like abilities-could revolutionize clinical MEG practice. Here, we developed and validated STIED, a simple yet powerful supervised DL algorithm combining two convolutional neural networks with temporal (1D time-course) and spatial (2D topography) features of MEG signals inspired from current clinical guidelines. Our DL model enabled both temporal and spatial localization of IEDs in patients suffering from focal epilepsy with frequent and high amplitude spikes (FE group), with high-performance metrics-accuracy, specificity, and sensitivity all exceeding 85%-when learning from spatiotemporal features of IEDs. This performance can be attributed to our handling of input data, which mimics established clinical MEG practice. Reverse engineering further revealed that STIED encodes fine spatiotemporal features of IEDs rather than their mere amplitude. The model trained on the FE group also showed promising results when applied to a separate group of presurgical patients with different types of refractory focal epilepsy, though further work is needed to distinguish IEDs from physiological transients. This study paves the way of incorporating STIED and DL algorithms into the routine clinical MEG evaluation of epilepsy.
2024-10-28	NeuGPT: Unified multi-modal Neural GPT	Yiqian Yang, Yiqun Duan, Hyejeong Jo, Qiang Zhang, Renjing Xu, Oiwi Parker Jones, Xuming Hu, Chin-teng Lin, Hui Xiong	Link	This paper introduces NeuGPT, a groundbreaking multi-modal language generation model designed to harmonize the fragmented landscape of neural recording research. Traditionally, studies in the field have been compartmentalized by signal type, with EEG, MEG, ECoG, SEEG, fMRI, and fNIRS data being analyzed in isolation. Recognizing the untapped potential for cross-pollination and the adaptability of neural signals across varying experimental conditions, we set out to develop a unified model capable of interfacing with multiple modalities. Drawing inspiration from the success of pre-trained large models in NLP, computer vision, and speech processing, NeuGPT is architected to process a diverse array of neural recordings and interact with speech and text data. Our model mainly focus on brain-to-text decoding, improving SOTA from 6.94 to 12.92 on BLEU-1 and 6.93 to 13.06 on ROUGE-1F. It can also simulate brain signals, thereby serving as a novel neural interface. Code is available at \href{https://github.com/NeuSpeech/NeuGPT}{NeuSpeech/NeuGPT (https://github.com/NeuSpeech/NeuGPT) .}
2024-10-25	Resolving Domain Shift For Representations Of Speech In Non-Invasive Brain Recordings	Jeremiah Ridge, Oiwi Parker Jones	Link	Machine learning techniques have enabled researchers to leverage neuroimaging data to decode speech from brain activity, with some amazing recent successes achieved by applications built using invasive devices. However, research requiring surgical implants has a number of practical limitations. Non-invasive neuroimaging techniques provide an alternative but come with their own set of challenges, the limited scale of individual studies being among them. Without the ability to pool the recordings from different non-invasive studies, data on the order of magnitude needed to leverage deep learning techniques to their full potential remains out of reach. In this work, we focus on non-invasive data collected using magnetoencephalography (MEG). We leverage two different, leading speech decoding models to investigate how an adversarial domain adaptation framework augments their ability to generalize across datasets. We successfully improve the performance of both models when training across multiple datasets. To the best of our knowledge, this study is the first ever application of feature-level, deep learning based harmonization for MEG neuroimaging data. Our analysis additionally offers further evidence of the impact of demographic features on neuroimaging data, demonstrating that participant age strongly affects how machine learning models solve speech decoding tasks using MEG data. Lastly, in the course of this study we produce a new open-source implementation of one of these models to the benefit of the broader scientific community.
2024-10-20	Non-invasive Neural Decoding in Source Reconstructed Brain Space	Yonatan Gideoni, Ryan Charles Timms, Oiwi Parker Jones	Link	Non-invasive brainwave decoding is usually done using Magneto/Electroencephalography (MEG/EEG) sensor measurements as inputs. This makes combining datasets and building models with inductive biases difficult as most datasets use different scanners and the sensor arrays have a nonintuitive spatial structure. In contrast, fMRI scans are acquired directly in brain space, a voxel grid with a typical structured input representation. By using established techniques to reconstruct the sensors' sources' neural activity it is possible to decode from voxels for MEG data as well. We show that this enables spatial inductive biases, spatial data augmentations, better interpretability, zero-shot generalisation between datasets, and data harmonisation.
2024-10-19	BrainECHO: Semantic Brain Signal Decoding through Vector-Quantized Spectrogram Reconstruction for Whisper-Enhanced Text Generation	Jilong Li, Zhenxi Song, Jiaqi Wang, Min Zhang, Zhiguo Zhang	Link	Recent advances in decoding language from brain signals (EEG and MEG) have been significantly driven by pre-trained language models, leading to remarkable progress on publicly available non-invasive EEG/MEG datasets. However, previous works predominantly utilize teacher forcing during text generation, leading to significant performance drops without its use. A fundamental issue is the inability to establish a unified feature space correlating textual data with the corresponding evoked brain signals. Although some recent studies attempt to mitigate this gap using an audio-text pre-trained model, Whisper, which is favored for its signal input modality, they still largely overlook the inherent differences between audio signals and brain signals in directly applying Whisper to decode brain signals. To address these limitations, we propose a new multi-stage strategy for semantic brain signal decoding via vEctor-quantized speCtrogram reconstruction for WHisper-enhanced text generatiOn, termed BrainECHO. Specifically, BrainECHO successively conducts: 1) Discrete autoencoding of the audio spectrogram; 2) Brain-audio latent space alignment; and 3) Semantic text generation via Whisper finetuning. Through this autoencoding--alignment--finetuning process, BrainECHO outperforms state-of-the-art methods under the same data split settings on two widely accepted resources: the EEG dataset (Brennan) and the MEG dataset (GWilliams). The innovation of BrainECHO, coupled with its robustness and superiority at the sentence, session, and subject-independent levels across public datasets, underscores its significance for language-based brain-computer interfaces.

neuroAI

Publish Date	Title	Authors	URL	Abstract
2024-11-27	NeuroAI for AI Safety	Patrick Mineault, Niccolò Zanichelli, Joanne Zichen Peng, Anton Arkhipov, Eli Bingham, Julian Jara-Ettinger, Emily Mackevicius, Adam Marblestone, Marcelo Mattar, Andrew Payne, Sophia Sanborn, Karen Schroeder, Zenna Tavares, Andreas Tolias	Link	As AI systems become increasingly powerful, the need for safe AI has become more pressing. Humans are an attractive model for AI safety: as the only known agents capable of general intelligence, they perform robustly even under conditions that deviate significantly from prior experiences, explore the world safely, understand pragmatics, and can cooperate to meet their intrinsic goals. Intelligence, when coupled with cooperation and safety mechanisms, can drive sustained progress and well-being. These properties are a function of the architecture of the brain and the learning algorithms it implements. Neuroscience may thus hold important keys to technical AI safety that are currently underexplored and underutilized. In this roadmap, we highlight and critically evaluate several paths toward AI safety inspired by neuroscience: emulating the brain's representations, information processing, and architecture; building robust sensory and motor systems from imitating brain data and bodies; fine-tuning AI systems on brain data; advancing interpretability using neuroscience methods; and scaling up cognitively-inspired architectures. We make several concrete recommendations for how neuroscience can positively impact AI safety.
2024-11-21	Evaluating Representational Similarity Measures from the Lens of Functional Correspondence	Yiqing Bo, Ansh Soni, Sudhanshu Srivastava, Meenakshi Khosla	Link	Neuroscience and artificial intelligence (AI) both face the challenge of interpreting high-dimensional neural data, where the comparative analysis of such data is crucial for revealing shared mechanisms and differences between these complex systems. Despite the widespread use of representational comparisons and the abundance classes of comparison methods, a critical question remains: which metrics are most suitable for these comparisons? While some studies evaluate metrics based on their ability to differentiate models of different origins or constructions (e.g., various architectures), another approach is to assess how well they distinguish models that exhibit distinct behaviors. To investigate this, we examine the degree of alignment between various representational similarity measures and behavioral outcomes, employing group statistics and a comprehensive suite of behavioral metrics for comparison. In our evaluation of eight commonly used representational similarity metrics in the visual domain -- spanning alignment-based, Canonical Correlation Analysis (CCA)-based, inner product kernel-based, and nearest-neighbor methods -- we found that metrics like linear Centered Kernel Alignment (CKA) and Procrustes distance, which emphasize the overall geometric structure or shape of representations, excelled in differentiating trained from untrained models and aligning with behavioral measures, whereas metrics such as linear predictivity, commonly used in neuroscience, demonstrated only moderate alignment with behavior. These insights are crucial for selecting metrics that emphasize behaviorally meaningful comparisons in NeuroAI research.
2024-10-25	A prescriptive theory for brain-like inference	Hadi Vafaii, Dekel Galor, Jacob L. Yates	Link	The Evidence Lower Bound (ELBO) is a widely used objective for training deep generative models, such as Variational Autoencoders (VAEs). In the neuroscience literature, an identical objective is known as the variational free energy, hinting at a potential unified framework for brain function and machine learning. Despite its utility in interpreting generative models, including diffusion models, ELBO maximization is often seen as too broad to offer prescriptive guidance for specific architectures in neuroscience or machine learning. In this work, we show that maximizing ELBO under Poisson assumptions for general sequence data leads to a spiking neural network that performs Bayesian posterior inference through its membrane potential dynamics. The resulting model, the iterative Poisson VAE (iP-VAE), has a closer connection to biological neurons than previous brain-inspired predictive coding models based on Gaussian assumptions. Compared to amortized and iterative VAEs, iP-VAElearns sparser representations and exhibits superior generalization to out-of-distribution samples. These findings suggest that optimizing ELBO, combined with Poisson assumptions, provides a solid foundation for developing prescriptive theories in NeuroAI.
2024-09-09	Evidence from fMRI Supports a Two-Phase Abstraction Process in Language Models	Emily Cheng, Richard J. Antonello	Link	Research has repeatedly demonstrated that intermediate hidden states extracted from large language models are able to predict measured brain response to natural language stimuli. Yet, very little is known about the representation properties that enable this high prediction performance. Why is it the intermediate layers, and not the output layers, that are most capable for this unique and highly general transfer task? In this work, we show that evidence from language encoding models in fMRI supports the existence of a two-phase abstraction process within LLMs. We use manifold learning methods to show that this abstraction process naturally arises over the course of training a language model and that the first "composition" phase of this abstraction process is compressed into fewer layers as training continues. Finally, we demonstrate a strong correspondence between layerwise encoding performance and the intrinsic dimensionality of representations from LLMs. We give initial evidence that this correspondence primarily derives from the inherent compositionality of LLMs and not their next-word prediction properties.
2024-07-22	Predictive Coding Networks and Inference Learning: Tutorial and Survey	Björn van Zwol, Ro Jefferson, Egon L. van den Broek	Link	Recent years have witnessed a growing call for renewed emphasis on neuroscience-inspired approaches in artificial intelligence research, under the banner of NeuroAI. A prime example of this is predictive coding networks (PCNs), based on the neuroscientific framework of predictive coding. This framework views the brain as a hierarchical Bayesian inference model that minimizes prediction errors through feedback connections. Unlike traditional neural networks trained with backpropagation (BP), PCNs utilize inference learning (IL), a more biologically plausible algorithm that explains patterns of neural activity that BP cannot. Historically, IL has been more computationally intensive, but recent advancements have demonstrated that it can achieve higher efficiency than BP with sufficient parallelization. Furthermore, PCNs can be mathematically considered a superset of traditional feedforward neural networks (FNNs), significantly extending the range of trainable architectures. As inherently probabilistic (graphical) latent variable models, PCNs provide a versatile framework for both supervised learning and unsupervised (generative) modeling that goes beyond traditional artificial neural networks. This work provides a comprehensive review and detailed formal specification of PCNs, particularly situating them within the context of modern ML methods. Additionally, we introduce a Python library (PRECO) for practical implementation. This positions PC as a promising framework for future ML innovations.
2023-10-29	Beyond Geometry: Comparing the Temporal Structure of Computation in Neural Circuits with Dynamical Similarity Analysis	Mitchell Ostrow, Adam Eisen, Leo Kozachkov, Ila Fiete	Link	How can we tell whether two neural networks utilize the same internal processes for a particular computation? This question is pertinent for multiple subfields of neuroscience and machine learning, including neuroAI, mechanistic interpretability, and brain-machine interfaces. Standard approaches for comparing neural networks focus on the spatial geometry of latent states. Yet in recurrent networks, computations are implemented at the level of dynamics, and two networks performing the same computation with equivalent dynamics need not exhibit the same geometry. To bridge this gap, we introduce a novel similarity metric that compares two systems at the level of their dynamics, called Dynamical Similarity Analysis (DSA). Our method incorporates two components: Using recent advances in data-driven dynamical systems theory, we learn a high-dimensional linear system that accurately captures core features of the original nonlinear dynamics. Next, we compare different systems passed through this embedding using a novel extension of Procrustes Analysis that accounts for how vector fields change under orthogonal transformation. In four case studies, we demonstrate that our method disentangles conjugate and non-conjugate recurrent neural networks (RNNs), while geometric methods fall short. We additionally show that our method can distinguish learning rules in an unsupervised manner. Our method opens the door to comparative analyses of the essential temporal structure of computation in neural circuits.
2023-05-25	Explaining V1 Properties with a Biologically Constrained Deep Learning Architecture	Galen Pogoncheff, Jacob Granley, Michael Beyeler	Link	Convolutional neural networks (CNNs) have recently emerged as promising models of the ventral visual stream, despite their lack of biological specificity. While current state-of-the-art models of the primary visual cortex (V1) have surfaced from training with adversarial examples and extensively augmented data, these models are still unable to explain key neural properties observed in V1 that arise from biological circuitry. To address this gap, we systematically incorporated neuroscience-derived architectural components into CNNs to identify a set of mechanisms and architectures that comprehensively explain neural activity in V1. We show drastic improvements in model-V1 alignment driven by the integration of architectural components that simulate center-surround antagonism, local receptive fields, tuned normalization, and cortical magnification. Upon enhancing task-driven CNNs with a collection of these specialized components, we uncover models with latent representations that yield state-of-the-art explanation of V1 neural activity and tuning properties. Our results highlight an important advancement in the field of NeuroAI, as we systematically establish a set of architectural components that contribute to unprecedented explanation of V1. The neuroscience insights that could be gleaned from increasingly accurate in-silico models of the brain have the potential to greatly advance the fields of both neuroscience and artificial intelligence.
2024-11-09	A Deep Probabilistic Spatiotemporal Framework for Dynamic Graph Representation Learning with Application to Brain Disorder Identification	Sin-Yee Yap, Junn Yong Loo, Chee-Ming Ting, Fuad Noman, Raphael C. -W. Phan, Adeel Razi, David L. Dowe	Link	Recent applications of pattern recognition techniques on brain connectome classification using functional connectivity (FC) are shifting towards acknowledging the non-Euclidean topology and dynamic aspects of brain connectivity across time. In this paper, a deep spatiotemporal variational Bayes (DSVB) framework is proposed to learn time-varying topological structures in dynamic FC networks for identifying autism spectrum disorder (ASD) in human participants. The framework incorporates a spatial-aware recurrent neural network with an attention-based message passing scheme to capture rich spatiotemporal patterns across dynamic FC networks. To overcome model overfitting on limited training datasets, an adversarial training strategy is introduced to learn graph embedding models that generalize well to unseen brain networks. Evaluation on the ABIDE resting-state functional magnetic resonance imaging dataset shows that our proposed framework substantially outperforms state-of-the-art methods in identifying patients with ASD. Dynamic FC analyses with DSVB-learned embeddings reveal apparent group differences between ASD and healthy controls in brain network connectivity patterns and switching dynamics of brain states. The code is available at https://github.com/Monash-NeuroAI/Deep-Spatiotemporal-Variational-Bayes.
2023-03-11	Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks	Feng-Lei Fan, Yingxin Li, Hanchuan Peng, Tieyong Zeng, Fei Wang	Link	Throughout history, the development of artificial intelligence, particularly artificial neural networks, has been open to and constantly inspired by the increasingly deepened understanding of the brain, such as the inspiration of neocognitron, which is the pioneering work of convolutional neural networks. Per the motives of the emerging field: NeuroAI, a great amount of neuroscience knowledge can help catalyze the next generation of AI by endowing a network with more powerful capabilities. As we know, the human brain has numerous morphologically and functionally different neurons, while artificial neural networks are almost exclusively built on a single neuron type. In the human brain, neuronal diversity is an enabling factor for all kinds of biological intelligent behaviors. Since an artificial network is a miniature of the human brain, introducing neuronal diversity should be valuable in terms of addressing those essential problems of artificial networks such as efficiency, interpretability, and memory. In this Primer, we first discuss the preliminaries of biological neuronal diversity and the characteristics of information transmission and processing in a biological neuron. Then, we review studies of designing new neurons for artificial networks. Next, we discuss what gains can neuronal diversity bring into artificial networks and exemplary applications in several important fields. Lastly, we discuss the challenges and future directions of neuronal diversity to explore the potential of NeuroAI.
2022-12-08	A Rubric for Human-like Agents and NeuroAI	Ida Momennejad	Link	Researchers across cognitive, neuro-, and computer sciences increasingly reference human-like artificial intelligence and neuroAI. However, the scope and use of the terms are often inconsistent. Contributed research ranges widely from mimicking behaviour, to testing machine learning methods as neurally plausible hypotheses at the cellular or functional levels, or solving engineering problems. However, it cannot be assumed nor expected that progress on one of these three goals will automatically translate to progress in others. Here a simple rubric is proposed to clarify the scope of individual contributions, grounded in their commitments to human-like behaviour, neural plausibility, or benchmark/engineering goals. This is clarified using examples of weak and strong neuroAI and human-like agents, and discussing the generative, corroborate, and corrective ways in which the three dimensions interact with one another. The author maintains that future progress in artificial intelligence will need strong interactions across the disciplines, with iterative feedback loops and meticulous validity tests, leading to both known and yet-unknown advances that may span decades to come.

medical

Publish Date	Title	Authors	URL	Abstract
2024-11-29	Neuroplasticity and Psychedelics: a comprehensive examination of classic and non-classic compounds in pre and clinical models	Claudio Agnorelli, Meg Spriggs, Kate Godfrey, Gabriela Sawicka, Bettina Bohl, Hannah Douglass, Andrea Fagiolini, Hashemi Parastoo, Robin Carhart-Harris, David Nutt, David Erritzoe	Link	Neuroplasticity, the ability of the nervous system to adapt throughout an organism's lifespan, offers potential as both a biomarker and treatment target for neuropsychiatric conditions. Psychedelics, a burgeoning category of drugs, are increasingly prominent in psychiatric research, prompting inquiries into their mechanisms of action. Distinguishing themselves from traditional medications, psychedelics demonstrate rapid and enduring therapeutic effects after a single or few administrations, believed to stem from their neuroplasticity-enhancing properties. This review examines how classic psychedelics (e.g., LSD, psilocybin, N,N-DMT) and non-classic psychedelics (e.g., ketamine, MDMA) influence neuroplasticity. Drawing from preclinical and clinical studies, we explore the molecular, structural, and functional changes triggered by these agents. Animal studies suggest psychedelics induce heightened sensitivity of the nervous system to environmental stimuli (meta-plasticity), re-opening developmental windows for long-term structural changes (hyper-plasticity), with implications for mood and behavior. Translating these findings to humans faces challenges due to limitations in current imaging techniques. Nonetheless, promising new directions for human research are emerging, including the employment of novel positron-emission tomography (PET) radioligands, non-invasive brain stimulation methods, and multimodal approaches. By elucidating the interplay between psychedelics and neuroplasticity, this review informs the development of targeted interventions for neuropsychiatric disorders and advances understanding of psychedelics' therapeutic potential.
2024-11-29	Graph Neural Networks for Heart Failure Prediction on an EHR-Based Patient Similarity Graph	Heloisa Oss Boll, Ali Amirahmadi, Amira Soliman, Stefan Byttner, Mariana Recamonde-Mendoza	Link	Objective: In modern healthcare, accurately predicting diseases is a crucial matter. This study introduces a novel approach using graph neural networks (GNNs) and a Graph Transformer (GT) to predict the incidence of heart failure (HF) on a patient similarity graph at the next hospital visit. Materials and Methods: We used electronic health records (EHR) from the MIMIC-III dataset and applied the K-Nearest Neighbors (KNN) algorithm to create a patient similarity graph using embeddings from diagnoses, procedures, and medications. Three models - GraphSAGE, Graph Attention Network (GAT), and Graph Transformer (GT) - were implemented to predict HF incidence. Model performance was evaluated using F1 score, AUROC, and AUPRC metrics, and results were compared against baseline algorithms. An interpretability analysis was performed to understand the model's decision-making process. Results: The GT model demonstrated the best performance (F1 score: 0.5361, AUROC: 0.7925, AUPRC: 0.5168). Although the Random Forest (RF) baseline achieved a similar AUPRC value, the GT model offered enhanced interpretability due to the use of patient relationships in the graph structure. A joint analysis of attention weights, graph connectivity, and clinical features provided insight into model predictions across different classification groups. Discussion and Conclusion: Graph-based approaches such as GNNs provide an effective framework for predicting HF. By leveraging a patient similarity graph, GNNs can capture complex relationships in EHR data, potentially improving prediction accuracy and clinical interpretability.
2024-11-29	MIMDE: Exploring the Use of Synthetic vs Human Data for Evaluating Multi-Insight Multi-Document Extraction Tasks	John Francis, Saba Esnaashari, Anton Poletaev, Sukankana Chakraborty, Youmna Hashem, Jonathan Bright	Link	Large language models (LLMs) have demonstrated remarkable capabilities in text analysis tasks, yet their evaluation on complex, real-world applications remains challenging. We define a set of tasks, Multi-Insight Multi-Document Extraction (MIMDE) tasks, which involves extracting an optimal set of insights from a document corpus and mapping these insights back to their source documents. This task is fundamental to many practical applications, from analyzing survey responses to processing medical records, where identifying and tracing key insights across documents is crucial. We develop an evaluation framework for MIMDE and introduce a novel set of complementary human and synthetic datasets to examine the potential of synthetic data for LLM evaluation. After establishing optimal metrics for comparing extracted insights, we benchmark 20 state-of-the-art LLMs on both datasets. Our analysis reveals a strong correlation (0.71) between the ability of LLMs to extracts insights on our two datasets but synthetic data fails to capture the complexity of document-level analysis. These findings offer crucial guidance for the use of synthetic data in evaluating text analysis systems, highlighting both its potential and limitations.
2024-11-29	SURE-VQA: Systematic Understanding of Robustness Evaluation in Medical VQA Tasks	Kim-Celine Kahl, Selen Erkan, Jeremias Traub, Carsten T. Lüth, Klaus Maier-Hein, Lena Maier-Hein, Paul F. Jaeger	Link	Vision-Language Models (VLMs) have great potential in medical tasks, like Visual Question Answering (VQA), where they could act as interactive assistants for both patients and clinicians. Yet their robustness to distribution shifts on unseen data remains a critical concern for safe deployment. Evaluating such robustness requires a controlled experimental setup that allows for systematic insights into the model's behavior. However, we demonstrate that current setups fail to offer sufficiently thorough evaluations, limiting their ability to accurately assess model robustness. To address this gap, our work introduces a novel framework, called SURE-VQA, centered around three key requirements to overcome the current pitfalls and systematically analyze the robustness of VLMs: 1) Since robustness on synthetic shifts does not necessarily translate to real-world shifts, robustness should be measured on real-world shifts that are inherent to the VQA data; 2) Traditional token-matching metrics often fail to capture underlying semantics, necessitating the use of large language models (LLMs) for more accurate semantic evaluation; 3) Model performance often lacks interpretability due to missing sanity baselines, thus meaningful baselines should be reported that allow assessing the multimodal impact on the VLM. To demonstrate the relevance of this framework, we conduct a study on the robustness of various fine-tuning methods across three medical datasets with four different types of distribution shifts. Our study reveals several important findings: 1) Sanity baselines that do not utilize image data can perform surprisingly well; 2) We confirm LoRA as the best-performing PEFT method; 3) No PEFT method consistently outperforms others in terms of robustness to shifts. Code is provided at https://github.com/IML-DKFZ/sure-vqa.
2024-11-29	Artifact Correction in Magnetic Resonance Temperature Imaging for Laser Interstitial Thermotherapy with Multi-echo Acquisitions	Ziyi Pan, Yuancheng Jiang, Wenbo Lv, Sisi Li, Meng Han, Yawei Kuang, Hao Sun, Xiu Wang, Jianjun Bai, Wenbo Liu, Guangzhi Wang, Hua Guo	Link	In MRI-guided laser interstitial thermotherapy (MRgLITT), a signal void sometimes appears at the heating center of the measured temperature map. In neurosurgical MRgLITT treatments, cerebrospinal fluid pulsation (CSF), which may lead to temperature artifacts, also needs to be carefully managed. We find that signal loss in MR magnitude images can be one distinct contributor to the temperature imaging signal void. Therefore, this study aims to investigate this finding and more importantly. Also, this study intends to improve measurement accuracy by correcting CSF-induced temperature errors and employing a more reliable phase unwrapping algorithm. A gradient echo sequence with certain TE values for temperature imaging is used to quantify T2* variations during MRgLITT and to investigate the development of signal voids throughout the treatment. Informed by these findings, a multi-echo GRE sequence with appropriate TE coverage is employed. A multi-echo-based correction algorithm is developed to address the signal loss-induced temperature errors. A new phase unwrapping method and a new CSF pulsation correction approach are developed for multi-echo signal processing. The temperature imaging method is evaluated by gel phantom, ex-vivo, and in-vivo LITT heating experiments. T2* shortening during heating can be one important cause of the temperate imaging signal voids and this demands the multi-echo acquisition with varied TE values. The proposed multi-echo-based method can effectively correct signal loss-induced temperature errors and raise temperature estimation precision. The multi-echo thermometry in the in-vivo experiments shows smoother hotspot boundaries, fewer artifacts, and improved thermometry reliability. In the in-vivo experiments, the ablation areas estimated from the multi-echo thermometry also show satisfactory agreement with those determined from post-ablation MR imaging.
2024-11-29	Contextual Checkerboard Denoise -- A Novel Neural Network-Based Approach for Classification-Aware OCT Image Denoising	Md. Touhidul Islam, Md. Abtahi M. Chowdhury, Sumaiya Salekin, Aye T. Maung, Akil A. Taki, Hafiz Imtiaz	Link	In contrast to non-medical image denoising, where enhancing image clarity is the primary goal, medical image denoising warrants preservation of crucial features without introduction of new artifacts. However, many denoising methods that improve the clarity of the image, inadvertently alter critical information of the denoised images, potentially compromising classification performance and diagnostic quality. Additionally, supervised denoising methods are not very practical in medical image domain, since a \emph{ground truth} denoised version of a noisy medical image is often extremely challenging to obtain. In this paper, we tackle both of these problems by introducing a novel neural network based method -- \emph{Contextual Checkerboard Denoising}, that can learn denoising from only a dataset of noisy images, while preserving crucial anatomical details necessary for image classification/analysis. We perform our experimentation on real Optical Coherence Tomography (OCT) images, and empirically demonstrate that our proposed method significantly improves image quality, providing clearer and more detailed OCT images, while enhancing diagnostic accuracy.
2024-11-29	SkelMamba: A State Space Model for Efficient Skeleton Action Recognition of Neurological Disorders	Niki Martinel, Mariano Serrao, Christian Micheloni	Link	We introduce a novel state-space model (SSM)-based framework for skeleton-based human action recognition, with an anatomically-guided architecture that improves state-of-the-art performance in both clinical diagnostics and general action recognition tasks. Our approach decomposes skeletal motion analysis into spatial, temporal, and spatio-temporal streams, using channel partitioning to capture distinct movement characteristics efficiently. By implementing a structured, multi-directional scanning strategy within SSMs, our model captures local joint interactions and global motion patterns across multiple anatomical body parts. This anatomically-aware decomposition enhances the ability to identify subtle motion patterns critical in medical diagnosis, such as gait anomalies associated with neurological conditions. On public action recognition benchmarks, i.e., NTU RGB+D, NTU RGB+D 120, and NW-UCLA, our model outperforms current state-of-the-art methods, achieving accuracy improvements up to $3.2\%$ with lower computational complexity than previous leading transformer-based models. We also introduce a novel medical dataset for motion-based patient neurological disorder analysis to validate our method's potential in automated disease diagnosis.
2024-11-29	Loop Shaping of Hybrid Motion Control with Contact Transition	Michael Ruderman	Link	A standard (stiff) motion control with output displacement feedback cannot handle unforeseen contact with environment without penetrating into soft, i.e. viscoelastic, materials or even damaging brittle or fragile materials. Robotics and mechatronics with tactile and haptic capabilities, and medical assistance systems in particular, place special demands on the advanced motion control systems that should enable safe and harmless contact transitions. This paper demonstrates how the fundamental principles of loop shaping can easily be used to handle the sufficiently stiff motion control with a sensor-free dynamic extension to reconfigure at contact with environment. Hybrid control scheme is proposed. Remarkable feature of the developed approach is that no measurement of the contact force is required and the input signal and measured output displacement are the only quantities used for control design and operation. Experimental scenarios for 1DOF actuator are shown where the moving tool comes into contact with grape fruits that are soft and penetrable at the same time.
2024-11-29	Adaptive Interactive Segmentation for Multimodal Medical Imaging via Selection Engine	Zhi Li, Kai Zhao, Yaqi Wang, Shuai Wang	Link	In medical image analysis, achieving fast, efficient, and accurate segmentation is essential for automated diagnosis and treatment. Although recent advancements in deep learning have significantly improved segmentation accuracy, current models often face challenges in adaptability and generalization, particularly when processing multi-modal medical imaging data. These limitations stem from the substantial variations between imaging modalities and the inherent complexity of medical data. To address these challenges, we propose the Strategy-driven Interactive Segmentation Model (SISeg), built on SAM2, which enhances segmentation performance across various medical imaging modalities by integrating a selection engine. To mitigate memory bottlenecks and optimize prompt frame selection during the inference of 2D image sequences, we developed an automated system, the Adaptive Frame Selection Engine (AFSE). This system dynamically selects the optimal prompt frames without requiring extensive prior medical knowledge and enhances the interpretability of the model's inference process through an interactive feedback mechanism. We conducted extensive experiments on 10 datasets covering 7 representative medical imaging modalities, demonstrating the SISeg model's robust adaptability and generalization in multi-modal tasks. The project page and code will be available at: [URL].
2024-11-28	Libra: Leveraging Temporal Images for Biomedical Radiology Analysis	Xi Zhang, Zaiqiao Meng, Jake Lever, Edmond S. L. Ho	Link	Radiology report generation (RRG) is a challenging task, as it requires a thorough understanding of medical images, integration of multiple temporal inputs, and accurate report generation. Effective interpretation of medical images, such as chest X-rays (CXRs), demands sophisticated visual-language reasoning to map visual findings to structured reports. Recent studies have shown that multimodal large language models (MLLMs) can acquire multimodal capabilities by aligning with pre-trained vision encoders. However, current approaches predominantly focus on single-image analysis or utilise rule-based symbolic processing to handle multiple images, thereby overlooking the essential temporal information derived from comparing current images with prior ones. To overcome this critical limitation, we introduce Libra, a temporal-aware MLLM tailored for CXR report generation using temporal images. Libra integrates a radiology-specific image encoder with a MLLM and utilises a novel Temporal Alignment Connector to capture and synthesise temporal information of images across different time points with unprecedented precision. Extensive experiments show that Libra achieves new state-of-the-art performance among the same parameter scale MLLMs for RRG tasks on the MIMIC-CXR. Specifically, Libra improves the RadCliQ metric by 12.9% and makes substantial gains across all lexical metrics compared to previous models.