显微镜即掩膜:冷冻电子断层扫描前向模型中的特权视角与标签
The microscope is the mask: privileged views and labels from a cryo-ET forward model
Bogdan Toader, Kiarash Jamali, Tanmay A. M. Bharat, Sjors H. W. Scheres
AI 速读 基于摘要
利用冷冻电子断层扫描前向模型生成的配对视角和模拟特权信息训练自监督模型CARNIVAL,在真实断层图像分类与检测任务中无需微调即优于对比学习基线。
cs.CV · cs.LG · math.OC
查看原文摘要
We explore the use of simulated data for training a model for protein annotation in crowded cryo-electron tomography volumes reconstructed from images collected at limited tilt angles and severely corrupted by the measurement operator. Firstly, we leverage the corruptions imposed by the forward model to generate domain-specific augmented paired views of the exact same scene for an invariance objective integrated into the LeJEPA self-supervised training framework. Secondly, we use additional information from the simulation pipeline such as the positions and identity of proteins in the simulated volumes to inform the architecture of the model and the loss function, so that semantic information is localised at protein positions in the resulting dense feature volume. The resulting model, CARNIVAL, is evaluated without finetuning on classification and detection tasks in real tomograms, using a benchmark dataset containing multiple protein types and two tomogram processing types. We show that CARNIVAL outperforms a state-of-the-art model trained using a contrastive objective on simulated data but without forward model-based paired views or privileged information.