跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.21477 · v1

Knowledge Fermentation: Learning from Shared Variation in Few-Shot Classification

知识发酵:从共享变异中学习的小样本分类

首次提交 2026-11-17 01:50
版本更新 2026-11-17 01:50
雷达观测 2026-11-17 16:10
arXiv ↗PDF ↗

摘要

A classifier built from a handful of labeled examples knows where each class sits in feature space but not how it varies, which is why prototype rules and linear probes overfit in the low-data regime. Knowledge Fermentation (KF) recovers this second-order knowledge by letting classes exchange what they know about variation. Residuals around class prototypes act as carriers; the variance of each direction is measured on classes that took no part in estimating it, and directions above the level of the surrounding medium define a structured noise that acts on a linear classifier as a closed-form change of geometry. A convex objective trains the classifier at a noise strength calibrated on the distances between prototypes, and unlabeled examples, when available, join as carriers in repeated rounds. We prove that the held-out variances are unbiased for the directions on which they are measured, that the fermented map never expands a direction, and that damping caused by wrong assignments limits itself. On 14 datasets from vision and language and 8 frozen encoders, with all constants fixed, KF is on par with the best of six tuned inductive baselines. With unlabeled carriers it raises one-shot accuracy from 52.4% to 65.8% on average and attains the highest average accuracy among nine transductive methods, significantly so on vision encoders, while information maximization stays ahead on sentence encoders. Controlled studies attribute the gain to the task-level geometry and show that it grows with the within-class variation that the classes of a task share.

AI 论文解读

基于摘要 · 非全文审阅

知识发酵(KF)通过让类别交换关于变异的知识,恢复二阶知识,从而提升小样本分类性能。

正在读取账号与研究记录…

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-17 01:50 · 最新观测版本

Knowledge Fermentation: Learning from Shared Variation in Few-Shot Classification

A classifier built from a handful of labeled examples knows where each class sits in feature space but not how it varies, which is why prototype rules and linear probes overfit in the low-data regime. Knowledge Fermentation (KF) recovers this second-order knowledge by letting classes exchange what they know about variation. Residuals around class prototypes act as carriers; the variance of each direction is measured on classes that took no part in estimating it, and directions above the level of the surrounding medium define a structured noise that acts on a linear classifier as a closed-form change of geometry. A convex objective trains the classifier at a noise strength calibrated on the distances between prototypes, and unlabeled examples, when available, join as carriers in repeated rounds. We prove that the held-out variances are unbiased for the directions on which they are measured, that the fermented map never expands a direction, and that damping caused by wrong assignments limits itself. On 14 datasets from vision and language and 8 frozen encoders, with all constants fixed, KF is on par with the best of six tuned inductive baselines. With unlabeled carriers it raises one-shot accuracy from 52.4% to 65.8% on average and attains the highest average accuracy among nine transductive methods, significantly so on vision encoders, while information maximization stays ahead on sentence encoders. Controlled studies attribute the gain to the task-level geometry and show that it grows with the within-class variation that the classes of a task share.

查看此版本解读 →查看此版本原文 ↗