← 研踪 PaperTrailarXiv:2611.11632 · v1
Perturbation Profiles of Scanpath Classifiers: What Reading, Code-Comprehension and Video Decoders Are Sensitive To
首次提交 2026-11-09 05:31
版本更新 2026-11-09 05:31
雷达观测 2026-11-10 13:19
摘要
When a scanpath descriptor discriminates a task, it is rarely clear what the discrimination rests on. We propose a perturbation profile for trial-level descriptors. Five transformations each destroy one stated property while preserving a listed set of invariants: fixation order, duration-position coupling, the duration distribution, orientation and handedness, and fine spatial structure. The classifier is refit, and the drop in participant-held-out AUC is the pipeline's sensitivity to that property. Because refitting lets the learner use redundant information, this describes the refitted pipeline, not what a fitted classifier used. Intervals come from a participant bootstrap that refits; in simulations at the sample sizes analysed, where the target was zero they excluded zero at a pooled rate of at most 2.5%, and they covered the target in 99.6% of 2720 cases. We profiled seven descriptors on twelve binary problems from three public datasets: reading (SB-SAT, 95 participants), program comprehension (EMIP, up to 159) and video and game viewing (GazeBase, 60). Permuting fixation order raised the AUC of a descriptor built on mean saccade amplitude on three problems (by up to 0.23), because permutation turns amplitude into a measure of spatial spread. A "comprehension" contrast that pooled two programs reached AUC 0.70 with a spread-only baseline and fell to 0.47 within one program: the profile located the confound in stimulus layout. On native versus non-native readers, recurrence was sensitive to order and to jitter at the scale of text lines but not of characters, while a two-feature duration and amplitude descriptor reached the same AUC (0.65). A graph descriptor built on the code's tokens was the strongest non-baseline descriptor on the program contrast (0.78; 0.87 with boosted trees). The interpreted results held under longer runs and three seeds; under a second learner 15 of 16 kept t
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-12 02:30 · 最新观测版本
Characterizing Scanpath Decoding Pipelines with Perturbation Profiles
When a scanpath descriptor supports a classification, it is rarely clear what the classification rests on. We propose perturbation profiles. Five transformations each target one stated property of every scanpath (fixation order, the coupling of durations to positions, the duration distribution, orientation and handedness, and fine spatial structure) while preserving a listed set of invariants. The classifier is refit, and the change in participant-held-out AUC is the pipeline's sensitivity to that property. Intervals come from a participant bootstrap that refits the classifier; in simulations they were conservative (99.7% coverage; standard errors about 1.8 times the empirical spread). We profiled nine descriptors on thirteen binary problems from three public datasets, with a within-fold AUC and class-balanced folds and a stated rule for which sensitivities to interpret. Of 565 intervals, 95 excluded zero; 80 were robust to the pooled AUC, 20 fold partitions and longer runs (36 of them within +/-0.05 AUC), and 45 also held with boosted trees. Most came from GazeBase (322 participants). Permuting fixation order raised the AUC of a descriptor based on mean saccade amplitude, because permuted amplitude has as its expectation the mean pairwise distance, a measure of spread. A "comprehension" contrast pooled over two programs was a contrast between their layouts: it vanished among trials of the same program, and synthetic gaze drawn from the layout alone reproduced the program contrast beneath it. For native versus non-native readers, spatial jitter at the scale of text lines reduced recurrence's discrimination. A Python package is released.
查看此版本解读 →查看此版本原文 ↗v1 · 2026-11-09 05:31
Perturbation Profiles of Scanpath Classifiers: What Reading, Code-Comprehension and Video Decoders Are Sensitive To
When a scanpath descriptor discriminates a task, it is rarely clear what the discrimination rests on. We propose a perturbation profile for trial-level descriptors. Five transformations each destroy one stated property while preserving a listed set of invariants: fixation order, duration-position coupling, the duration distribution, orientation and handedness, and fine spatial structure. The classifier is refit, and the drop in participant-held-out AUC is the pipeline's sensitivity to that property. Because refitting lets the learner use redundant information, this describes the refitted pipeline, not what a fitted classifier used. Intervals come from a participant bootstrap that refits; in simulations at the sample sizes analysed, where the target was zero they excluded zero at a pooled rate of at most 2.5%, and they covered the target in 99.6% of 2720 cases. We profiled seven descriptors on twelve binary problems from three public datasets: reading (SB-SAT, 95 participants), program comprehension (EMIP, up to 159) and video and game viewing (GazeBase, 60). Permuting fixation order raised the AUC of a descriptor built on mean saccade amplitude on three problems (by up to 0.23), because permutation turns amplitude into a measure of spatial spread. A "comprehension" contrast that pooled two programs reached AUC 0.70 with a spread-only baseline and fell to 0.47 within one program: the profile located the confound in stimulus layout. On native versus non-native readers, recurrence was sensitive to order and to jitter at the scale of text lines but not of characters, while a two-feature duration and amplitude descriptor reached the same AUC (0.65). A graph descriptor built on the code's tokens was the strongest non-baseline descriptor on the program contrast (0.78; 0.87 with boosted trees). The interpreted results held under longer runs and three seeds; under a second learner 15 of 16 kept t
查看此版本解读 →查看此版本原文 ↗