通过分层脑电-文本对齐进行思维解码
Thought-deciphering via hierarchical EEG-to-text alignment
Xiaopeng Mao, Morten Mørup, Xin Xu, Sang Wan Lee, Sadasivan Puthusserypady
AI 速读 基于摘要
提出分层脑电-文本对齐框架,先对脑电做粗粒度语义分类再在类内检索句子,从而从未见过的句子中实现句子级脑电解码。
cs.LG · eess.SP
查看原文摘要
Non-invasive brain-to-text interfaces offer alternative communication pathways for people with communication disabilities. However, sentence-level decoding from electroencephalography (EEG) remains challenging. We propose a hierarchical EEG-text alignment framework that retrieves previously unseen sentences by progressively decoding coarse-grained semantic information and fine-grained sentence representations. We hypothesize that broader semantic context can be more reliably captured from EEG than individual sentences. Accordingly, EEG representations are first classified into semantic classes, which are defined as groups of related sentences, followed by sentence retrieval within the predicted class. Among over 2,000 unique text samples, our approach achieves, on unseen sentences, average top-5 and top-10 retrieval accuracies of 62.6\% $\pm$ 1.0\% and 83.5\% $\pm$ 1.0\%, respectively, compared with 0.3\% $\pm$ 0.3\% and 0.5\% $\pm$ 0.2\% for direct EEG-sentence alignment without semantic classification. Qualitative analysis further reveals that decoding performance depends strongly on textual context. These findings demonstrate the potential of hierarchical semantic alignment for advancing sentence-level EEG-to-text decoding.