CLARiTy: A Vision Transformer for Multi-Label Classification and Weakly-Supervised Localization of Chest X-ray Pathologies
John M. Statheros, Hairong Wang, Richard Klein
原文摘要
Chest X-ray interpretation requires multi-label classification and spatial localization, yet region-level annotations are scarce, so weakly supervised localization involves training on image-level labels to produce class heatmaps. The step that converts heatmaps into boxes is rarely calibrated, that is, fitted on validation images, and localization results seldom leave the training dataset. We present CLARiTy (Class Localizing and Attention Refining Image Transformer), a vision transformer whose class tokens yield class-specific attention maps without gradient class activation mapping. A SegmentCAM head trained on anatomical region masks, not pathology masks, suppresses background. OneBox then converts the heatmaps into boxes, with parameters fitted per finding and resolution. On the official NIH split, CLARiTy outperforms the compared methods at matched resolution in localization and remains competitive in classification. Against published ThoraX-PriorNet values at matched 224 px resolution, the relative gain in macro intersection-over-union (IoU) accuracy at IoU 0.5 is 50.7%. The largest relative gain is on nodules: at that threshold the 224-pixel model reaches 0.154, where no compared method exceeds 0.013. Without training any weights on RSNA Pneumonia or VinDr-CXR, we refit only the OneBox parameters on target validation images, leaving the model and heatmaps unchanged. Localization is competitive with RAD-DINO and ahead of TorchXRayVision DenseNet121-NIH at every threshold. Fitting the box extractor on validation images, freezing it before testing and reporting localization across thresholds gives a protocol that carries to new corpora without retraining the model. Code is available at https://github.com/jstat17/clarity-cxr.
cs.LG