Segmentation foundation models offer reusable tools for universal medical-image segmentation, but their development and evaluation have largely centered on structural CT and MRI imaging. Functional imaging such as PET provides complementary metabolic information yet remains less studied and validated. Moreover, existing evaluations often involve different datasets and tasks, making modality-associated variation difficult to distinguish from differences in segmentation targets and data distributions. To enable a more direct comparison, we introduce UMD, a whole-body multi-organ dataset comprising 490 PET/CT and 464 PET/MRI examinations with 13 annotated organs (approximately 675,000 2D slices and 12,000 3D organ annotations). UMD evaluates structural and functional images separately on corresponding organs within the same examination, reducing variation in patient anatomy and target selection. We benchmark the out-of-the-box segmentation capabilities of five representative 3D foundation models on UMD. Overall, most evaluated models show limited PET segmentation performance, and some configurations also yield near-zero overlap on CT or MRI under the evaluated workflows. These findings highlight the need to assess tool applicability across different imaging conditions and identify opportunities for cross-modality generalization. UMD provides a paired data resource and evaluation setting for this investigation.
KV cache reuse accelerates LLM serving, but existing systems manage host DRAM and NVMe SSD as a tiered hierarchy where DRAM acts as an intermediate cache. Production LLM workloads break both premises of this hierarchy. First, bimodal reuse defeats DRAM temporal filtering, since workloads are dominated either by one-shot blocks with long reuse distances or by multi-gigabyte agentic sessions that exceed DRAM capacity, putting SSD on the critical read path. Second, DRAM staging and eviction-ordered layout bottleneck SSD restoration. Holding host staging buffers for in-flight prefixes causes contention under load that aborts prefetches, while incremental eviction scatters a session's blocks across the SSD and prevents large sequential reads. We present FlatCake, a flat KV storage architecture that exposes host DRAM and NVMe SSD as peer media in a unified 64-bit address space. Exploiting the fact that KV blocks are write-once, immutable, and reused together within a prompt or session, FlatCake places each block directly onto its target medium in session-contiguous, layer-major order at creation time and never migrates it. FlatCake streams coalesced SSD reads directly into a bounded GPU window via GPUDirect Storage and decouples DRAM and SSD request queues to prevent head-of-line blocking. We implement FlatCake in SGLang and compare it against host-staged (Mooncake) and GDS-enabled (Mooncake-GDS) tiered storage, as well as SSD-direct storage (Tutti). Across three production agentic, coding, and conversational traces with DeepSeek-V4-Flash, FlatCake reduces mean TTFT near the baselines' latency knees by $1.40\times$--$2.42\times$ over the fastest baseline. On the agentic Claude Code trace, FlatCake sustains a 91.6% cache hit rate under load (versus 71.1% for all three baselines) and delivers up to $2.16\times$ higher token throughput.
Structure--Residual Inductive Biases for Language Model Pretraining
Hai Huang, Bartosz Żuk, Rassul Magauin, Jiaxin Huang, Xue Liu
AI 速读 基于摘要
作者提出结构-残差归纳偏置,通过边界瓶颈概念和连续概念两种方法,在语言模型预训练中改进下一词预测。
cs.CL · cs.AI
查看原文摘要
Next-token prediction is relied upon to learn both predictive behavior and internal representations. We ask whether representation structure itself should be an explicit inductive bias. Motivated by the two-part coding view of Kolmogorov complexity, we study a structure--residual principle in which a representation separates structure from realization detail. We instantiate this principle in two complementary ways. Boundary Bottleneck Concept (BBC) routes information from token segments through boundary carriers while preserving local autoregressive computation. Continuous Concept (CC) regularizes representation geometry and induces a spectral decomposition into low-frequency structure and high-frequency residual. We derive coding bounds for both views and show that multiplicity-corrected boundary attention becomes exact as the within-segment path energy vanishes. Empirically, BBC and CC exhibit strikingly different depth preferences: BBC works best in early layers, whereas CC is most effective in later layers. At 300M training tokens, combining BBC and CC improves NLL by 1.44\% on GPT-2 Medium. These gains persist with training runs up to 20B tokens. Representation diagnostics show that BBC increases the relative predictive importance of boundary states, while CC reduces spectral residual energy. Together, these results suggest that structure--residual biases can improve language-model pretraining beyond next-token prediction. Code: https://github.com/haihuang-ml/structure-residual