← 研踪 PaperTrailarXiv:2611.10033 · v1
Ultra-Lightweight 3D Medical Image Segmentation via Knowledge Distillation in Label-Scarce Domains
首次提交 2026-11-05 21:20
版本更新 2026-11-05 21:20
雷达观测 2026-11-10 10:12
摘要
Mamba is a neural network architecture that reduces computational complexity, and has been shown to be effective for 3D medical image segmentation. However, existing models still remain computationally expensive. We propose compressing Mamba by distillation while achieving competitive performance. Specifically, we demonstrate compression of a 139M-parameter SegMamba-V2 teacher into a lightweight TinyUNet3D student with about 0.96M parameters, achieving about 145$\times$ compression with a performance drop of just over one Dice point. We further investigate the utility of knowledge distillation in a range of data-labeling scenarios under identical experimental settings, and show that its advantage is most prominent when labeled data is scarce. To demonstrate the generalizability of our approach, experiments are conducted on three datasets: BraTS 2023, MSD Spleen and MSD Hippocampus. The code for TinyUNet3D is publicly available at: https://github.com/2802990u/tinyunet3d-dist-kd.
作者备注：This is a four pages paper having 3 figures and is submitted to ISBI2027
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-05 21:20 · 最新观测版本
Ultra-Lightweight 3D Medical Image Segmentation via Knowledge Distillation in Label-Scarce Domains
Mamba is a neural network architecture that reduces computational complexity, and has been shown to be effective for 3D medical image segmentation. However, existing models still remain computationally expensive. We propose compressing Mamba by distillation while achieving competitive performance. Specifically, we demonstrate compression of a 139M-parameter SegMamba-V2 teacher into a lightweight TinyUNet3D student with about 0.96M parameters, achieving about 145$\times$ compression with a performance drop of just over one Dice point. We further investigate the utility of knowledge distillation in a range of data-labeling scenarios under identical experimental settings, and show that its advantage is most prominent when labeled data is scarce. To demonstrate the generalizability of our approach, experiments are conducted on three datasets: BraTS 2023, MSD Spleen and MSD Hippocampus. The code for TinyUNet3D is publicly available at: https://github.com/2802990u/tinyunet3d-dist-kd.
查看此版本解读 →查看此版本原文 ↗