MCL: Multi-Domain Feature-Driven Meta-learning for Subject-Independent Assessment of Cognitive Tasks Using Hybrid EEG–fNIRS

Reference implementation of the Meta-learning Cross-subject Learner (MCL), a compact model for subject-independent classification of cognitive tasks from hybrid EEG–fNIRS recordings. A classifier trained on one group of people usually degrades sharply on a new person, because neural and hemodynamic responses vary between individuals. Instead of committing to a single fixed decision boundary, MCL learns an initialization that adapts to each new subject from a few examples, driven by a task-informed, multi-domain feature set.

The work has two stages: a systematic multi-domain feature benchmark across the time, frequency, and time–frequency domains that identifies the most transferable descriptors (frequency-domain shape, FDS, and temporal dynamics, TDT), followed by the MCL model that couples a dual-stream encoder with a nested meta-iteration for cross-subject adaptation.

Keywords

Subject-independent classification, multi-domain feature analysis, meta-learning, hybrid EEG–fNIRS, cognitive task assessment, brain–computer interface, cross-subject generalization.

Highlights

• Subject-independent decoding. Cross-subject accuracies of 84.5% (n-back), 96.6% (DSR), and 94.8% (word generation), well above convolutional, dense, and recurrent baselines that collapse under inter-subject variability.
• Multi-domain feature benchmark. Six feature groups (TDT, TDS, TDM, FDS, FDP, TFD) compared task by task, with FDS and TDT carried forward as the most discriminative and transferable.
• Meta-learning core. An inner loop adapts the shared parameters to each subject, and an outer loop learns an initialization that transfers to unseen subjects.
• Lightweight. About 55k parameters, small enough for real-time brain–computer interface use.

Method overview

MCL processes each modality with its own stream:

1. Dual-stream CNN–LSTM encoder. The EEG and fNIRS branches are read separately by stacked convolutional blocks (spatial structure) followed by an LSTM (temporal structure), then fused by concatenation.
2. Representation-level fusion. The two learned embeddings are concatenated and passed to a dense classification head with a soft-max output.
3. Nested meta-iteration. Tasks are subject-level support/query splits. The inner loop adapts to a sampled subject in a few gradient steps; the outer loop updates the shared initialization on the query loss, so adaptation carries over to subjects the model has not seen.

Repository structure

Core model and training:

• Config.py: shared constants (tasks, feature groups, signal settings, shapes, hyperparameters)
• ModelArchitecture.py: the dual-stream CNN–LSTM architecture and the inner/outer meta-learning loops
• ModelInitialization.py: builds the model, task distribution, and optimizers
• TrainingLoop.py: the meta-iteration driver (inner loop then outer loop)
• DataAugmentation.py: augmentation pipeline for EEG–fNIRS feature maps

Features, baselines, and analysis:

• FeatureExtraction.py: the six multi-domain feature groups (pure NumPy/SciPy)
• Baselines.py: Conv / Dense / Recur baselines for the exploration stage
• Metrics.py: accuracy/F1, cross-subject evaluation, and paired significance tests
• Complexity.py: parameter count, FLOPs, and asymptotic time complexity
• Visualization.py: t-SNE, confusion-matrix, and per-subject accuracy plots

Requirements

• Python 3.9 or newer
• TensorFlow 2.x, NumPy, SciPy, scikit-learn, Matplotlib, imgaug

Install with:

pip install -r requirements.txt

Data

The experiments use the public hybrid EEG–fNIRS dataset of Shin et al. (26 participants; n-back, DSR, and word-generation tasks), available at https://doc.ml.tu-berlin.de/simultaneous_EEG_NIRS/.

Download and extract it, then point MCL at it with an environment variable (defaults to ./data):

export MCL_DATA=/path/to/eeg_fnirs_dataset

EEG is band-pass filtered 1–45 Hz and fNIRS 0.01–0.2 Hz, then segmented into sliding windows. After feature extraction each trial is represented as an EEG tensor of shape (18, 360, 1) and an fNIRS tensor of shape (18, 72, 1).

Usage

1. Extract multi-domain features

import numpy as np
from FeatureExtraction import extract, GROUPS

# x: a windowed signal of shape (n_frames, n_samples)
feats_best = extract(x, groups=("FDS", "TDT"), fs=200)  # the two carried forward
feats_all  = extract(x, groups=tuple(GROUPS), fs=200)    # full benchmark

2. Explore with the deep baselines

from Baselines import BASELINES
from Config import EEG_SHAPE, NIRS_SHAPE, TASKS

model = BASELINES["Conv"](EEG_SHAPE, NIRS_SHAPE, num_classes=TASKS["nback"])
model.compile(optimizer="adam", loss="categorical_crossentropy", metrics=["accuracy"])
model.fit([eeg_train, nirs_train], y_train, epochs=150, batch_size=32)

Swap "Conv" for "Dense" or "Recur" to reproduce the other baselines.

3. Train MCL (meta-iteration)

ModelInitialization.py builds the model, task distribution, and optimizers, and TrainingLoop.py runs the nested meta-iteration. With the dataset in place and the feature tensors prepared:

python ModelInitialization.py   # build model, tasks, optimizers
python TrainingLoop.py          # inner-loop adaptation + outer-loop meta-update

Training settings (100 meta-iterations, 5 inner steps, 10 tasks, learning rate 1e-4) are defined in Config.py.

4. Evaluate cross-subject performance

from Metrics import evaluate_cross_subject, significance_vs_reference

acc, f1 = evaluate_cross_subject(model, by_subject_data, test_subjects)

# per_subject: {"MCL": [...], "Conv": [...], ...} of per-subject accuracies
sig = significance_vs_reference(per_subject, reference="MCL", method="wilcoxon")

5. Report complexity and make figures

from Complexity import summary
summary(model)   # asymptotic complexity, parameter count, FLOPs

from Visualization import extract_embeddings, plot_tsne, plot_confusion
emb = extract_embeddings(model, [eeg_test, nirs_test])
plot_tsne(emb, labels, class_names=["0-back", "2-back", "3-back"])

Results

Cross-subject performance (subject-level hold-out):

Task	Accuracy	F1-score
n-back	84.5%	84.2%
DSR	96.6%	94.7%
Word generation	94.8%	94.5%

All gains over the convolutional, dense, and recurrent baselines are significant at p < 0.001 (paired test across subjects).

Citation

If you find this work useful, please cite our article:

Contact

Questions and issues are welcome through the repository's issue tracker.