跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.03623 · v2

Thought-deciphering via hierarchical EEG-to-text alignment

Xiaopeng MaoMorten MørupXin XuSang Wan LeeSadasivan Puthusserypady
cs.LGeess.SP
首次提交 2026-11-03 02:52
版本更新 2026-11-04 04:45
雷达观测 2026-11-05 10:08
arXiv ↗PDF ↗

摘要

Non-invasive brain-to-text interfaces offer alternative communication pathways for people with communication disabilities. However, sentence-level decoding from electroencephalography (EEG) remains challenging. We propose a hierarchical EEG-text alignment framework that retrieves sentences by progressively decoding coarse-grained semantic information and fine-grained sentence representations. We hypothesize that broader semantic context can be more reliably captured from EEG than individual sentences. Accordingly, EEG representations are first aligned with semantic classes, followed by sentence retrieval within the predicted class. Among over 2,000 unique text samples, our approach achieves average top-5 and top-10 retrieval accuracies of 62.6\% $\pm$ 1.0\% and 83.5\% $\pm$ 1.0\%, respectively, compared with 50.3\% $\pm$ 2.3\% and 68.3\% $\pm$ 2.0\% for direct EEG-sentence alignment. Qualitative analysis further reveals that decoding performance depends strongly on textual context. These findings demonstrate the potential of hierarchical semantic alignment for improving sentence-level EEG-to-text decoding.

查看 arXiv 原文 ↗阅读 PDF ↗

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存，登录后共享已有结果。

版本差异

仅对比实际观测的元数据，不代表 PDF 正文或实验变化。版本可能不连续；带删除线为移除，下划线为新增。

摘要变化

Communication is important for our well-being, but it can be impaired by disabilities, such as locked-in syndrome (LIS), cerebral palsy (CP), and dysarthria. Non-invasive brain-to-text brain-computer interfaces (BCIs) offer a promising solution by providing an alternative communication pathwaypathways for people with communication disabilities. However, most recent breakthroughs have relied on large, cumbersome, and costly neuroimaging modalities, such as functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). Numerous studies have attemptedsentence-level decoding using the lighter and cheaperfrom electroencephalography (EEG), but the results remain inferior compared to the otherremains modalitieschallenging. More importantly, weWe dopropose nota understandhierarchical howEEG-text thesealignment modalitiesframework arethat relatedretrieves tosentences textby atprogressively thedecoding letter,coarse-grained word,semantic sentence,information and higherfine-grained semanticsentence levelsrepresentations. Many prior studies have focused on decodingWe belowhypothesize thethat sentencebroader levelsemantic becausecontext itcan isbe considerablymore easierreliably tocaptured designfrom anEEG experimentthan aroundindividual phonemes,sentences. lettersAccordingly, andEEG words.representations However,are duringfirst communication,aligned sentenceswith playsemantic aclasses, muchfollowed moreby importantsentence roleretrieval thanwithin wordsthe andpredicted lettersclass. This isAmong becauseover sentences2,000 carryunique informationtext atsamples, suchour aapproach highachieves levelaverage thattop-5 theyand cantop-10 beretrieval understoodaccuracies evenof if62.6\% some$\pm$ words1.0\% and letters83.5\% are$\pm$ missing1.0\%, Howeverrespectively, withincompared sentence-levelwith decoding,50.3\% sentence$\pm$ classification2.3\% isand not68.3\% as$\pm$ well2.0\% investigatedfor asdirect EEG-sentence generation,alignment. makingQualitative itanalysis difficultfurther toreveals decidethat whetherdecoding aperformance sentencedepends shouldstrongly beon retrievedtextual orcontext. generatedThese infindings demonstrate the contextpotential of EEG-to-text decoding. Thishierarchical papersemantic strivesalignment tofor investigateimproving sentence-level EEG-to-text classification via a novel hierarchical alignment framework, where the main goal is to retrieve a read sentence using its context and EEG as inputsdecoding.

观测到的版本

相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-04 04:45 · 最新观测版本

Thought-deciphering via hierarchical EEG-to-text alignment

Non-invasive brain-to-text interfaces offer alternative communication pathways for people with communication disabilities. However, sentence-level decoding from electroencephalography (EEG) remains challenging. We propose a hierarchical EEG-text alignment framework that retrieves sentences by progressively decoding coarse-grained semantic information and fine-grained sentence representations. We hypothesize that broader semantic context can be more reliably captured from EEG than individual sentences. Accordingly, EEG representations are first aligned with semantic classes, followed by sentence retrieval within the predicted class. Among over 2,000 unique text samples, our approach achieves average top-5 and top-10 retrieval accuracies of 62.6\% $\pm$ 1.0\% and 83.5\% $\pm$ 1.0\%, respectively, compared with 50.3\% $\pm$ 2.3\% and 68.3\% $\pm$ 2.0\% for direct EEG-sentence alignment. Qualitative analysis further reveals that decoding performance depends strongly on textual context. These findings demonstrate the potential of hierarchical semantic alignment for improving sentence-level EEG-to-text decoding.

查看此版本解读 →查看此版本原文 ↗
v1 · 2026-11-03 02:52

Thought-deciphering via hierarchical EEG-to-text alignment

Communication is important for our well-being, but it can be impaired by disabilities, such as locked-in syndrome (LIS), cerebral palsy (CP), and dysarthria. Non-invasive brain-to-text brain-computer interfaces (BCIs) offer a promising solution by providing an alternative communication pathway for people with communication disabilities. However, most recent breakthroughs have relied on large, cumbersome, and costly neuroimaging modalities, such as functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG). Numerous studies have attempted decoding using the lighter and cheaper electroencephalography (EEG), but the results remain inferior compared to the other modalities. More importantly, we do not understand how these modalities are related to text at the letter, word, sentence, and higher semantic levels. Many prior studies have focused on decoding below the sentence level because it is considerably easier to design an experiment around phonemes, letters, and words. However, during communication, sentences play a much more important role than words and letters. This is because sentences carry information at such a high level that they can be understood even if some words and letters are missing. However, within sentence-level decoding, sentence classification is not as well investigated as sentence generation, making it difficult to decide whether a sentence should be retrieved or generated in the context of EEG-to-text decoding. This paper strives to investigate sentence-level EEG-to-text classification via a novel hierarchical alignment framework, where the main goal is to retrieve a read sentence using its context and EEG as inputs.

查看此版本解读 →查看此版本原文 ↗