ArchSpace:记录、评审与引用 LLM 架构研究
ArchSpace: Recording, Reviewing, and Referencing LLM Architecture Research
Jie Wang, Tong Zhu, Jiahao Kuang, Zhe Xu, Mowen Ruan, Yunkai Chai, Yuliang Liu, Dehong Zheng 等 14 位作者
AI 速读 基于摘要
ArchSpace 是一个公开平台,用于记录 LLM 架构提案及其实验与推理、由社区评审,并供后续工作引用,同时借助缩放定律在小规模实验上预测更大规模的模型损失。
cs.CL · cs.LG
查看原文摘要
Model architecture shapes the scaling behavior, training efficiency, and inference cost of LLMs, making it a central focus of both industry and academia. In industry, teams such as those behind DeepSeek, Qwen, and Kimi routinely introduce architectural innovations, yet public reports rarely document which ideas succeeded, which were abandoned, and why. In academia, limited compute and inconsistent baselines often restrict evaluations to small-scale, single-point comparisons, making it difficult to establish whether improvements hold at larger scales or translate into broader impact. ArchSpace addresses both gaps as a public platform for Recording, Reviewing, and Referencing Architecture Research: each proposal is recorded with its experiments and reasoning, reviewed by the community, and referenced by later work. To evaluate ideas before costly large-scale training, ArchSpace leverages scaling laws to predict model loss at larger scales from smaller experiments, a practice widely adopted in industry. We provide initial references that make the platform usable: Qwen3.5 and OLMo3 baselines, fitted scaling laws, and a reusable training recipe. Ideas validated through this review can in turn become new baselines, and ArchSpace thereby aims to broaden the range of ideas that shape the next generation of models.