提出 Metal Sandbox 基础设施,通过隔离文件访问、原生命令执行与跨 Mac 的命令级调度,减少 GPU 编码智能体 rollout 中的 GPU 空闲并提升命令吞吐。
cs.LG · cs.AI · cs.CL · cs.DC
查看原文摘要
On-device inference requires kernels that make effective use of local accelerators. Efficient training and evaluation of models that generate Apple Metal kernels benefit from native macOS tools and concurrent agent sessions. These sessions alternate between model requests, development commands, and GPU execution, leaving GPU capacity idle for much of a rollout. We present Metal Sandbox, infrastructure that combines restricted per-instance file access, native command execution, and command-level scheduling across interconnected Macs. Each agent retains its workspace and private development environment. Parallel commands can run concurrently, while serial commands share one execution slot per node; eligible waiting commands can migrate to available Macs and return results to the original agent. Integration with Harbor supports task execution, automatic verification, and rollout collection through a common workflow. Controlled experiments on four Mac mini M4 nodes achieve $1.68\times$--$2.05\times$ gains in command throughput under concentrated demand. Sixteen static agent traces give an estimated $2.14\times$ sustained-throughput gain for four sessions per node versus one, assuming continuously replenished sessions and unchanged mean durations. Replaying these measured traces under dynamic scheduling reduces mean tool waiting by 18.8% and batch elapsed time by 22.3%, assuming unchanged phase durations. Metal Sandbox and its Harbor integration are open-source on GitHub.
Qixin Xu, Haozhe Wang, Che Liu, Fangzhen Lin, Wenhu Chen
原文摘要
Answering questions over long, visually rich documents demands both cross-page structural reasoning and fine-grained visual understanding. Existing approaches sacrifice one for the other: retrieval-augmented methods fragment documents and lose inter-page structure; monolithic vision-language models preserve structure but scale poorly beyond a few pages. This impasse stems from a false premise: that the macroscopic task of evidence retrieval and the microscopic task of detailed reasoning must either be split into separate systems or be processed identically. By decoupling their perceptual resolutions rather than their architectures, a single model can achieve the structural holism of monolithic processing alongside the efficiency of targeted retrieval. UniDoc realizes this through two resolution-adaptive stages: a Fast Reading stage processes all pages at low resolution to structurally identify evidence pages, and a Focused Thinking stage re-processes only those pages at high resolution for multimodal inference. To train this pipeline, we construct 30K multi-hop reasoning instances with stage-level supervision. On MMLongBench-Doc, UniDoc yields relative accuracy gains of 55.7% on Qwen2.5-VL-7B and 16.9% on Qwen3-VL-8B, while significantly reducing token cost relative to uniform high-resolution processing. Error analysis identifies page retrieval and fine-grained visual reasoning as the remaining bottlenecks, delineating concrete targets for future work.