← 研踪 PaperTrailarXiv:2611.14984 · v1
VRMaaS: Verifiable Reward-Model-as-a-Service for Trustworthy LLM Alignment
首次提交 2026-11-11 11:41
版本更新 2026-11-11 11:41
雷达观测 2026-11-13 10:09
摘要
Reward models (RMs) are increasingly deployed as service components in LLM alignment pipelines, where payment, audit, moderation, and governance workflows act on the scores they return. An inference proof alone does not show whether a score belongs to the intended request, model checkpoint, numerical configuration, and verifier: in a controlled comparison that isolates the accepted statement from prover scale, a generic inference proof wrapped for on-chain checking remains valid after six classes of such service-field substitution. We formulate verifiable reward-model-as-a-service (VRMaaS) as a service-output integrity problem and present a framework in which each returned reward becomes a proof-carrying artifact. Its off-chain acceptance rule binds request identity, committed weights, verifier and configuration identifiers, proof-task order, and the digest of the exported package, so auditors can check a reward without rerunning the private RM or exposing inputs, checkpoints, or hidden activations; an optional on-chain zero-knowledge wrapper gives contracts a partial attestation of selected consistency checks and supplied service fields. The framework decomposes Transformer RM serving into arithmetic and lookup-backed proof tasks, aggregates them into a composite proof, and links hidden operands to the committed weights. A GPU prototype on 1B, 3B, and 8B reward models over 128-1024 tokens preserves 93.5-99% pairwise agreement with the baseline RM and rejects every tested service-field substitution in complete artifacts. Using exact integer witnesses and request-level aggregation of weight-binding openings, it generates complete artifacts in 49.8-206.1 seconds across the evaluated settings; auditors verify retained artifacts in 1.94-5.11 seconds, and partial public attestation costs about 217K gas. These costs suit delayed audit, settlement, and governance workflows.
作者备注：24 pages, 6 figures, including appendices A-O
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-11 11:41 · 最新观测版本
VRMaaS: Verifiable Reward-Model-as-a-Service for Trustworthy LLM Alignment
Reward models (RMs) are increasingly deployed as service components in LLM alignment pipelines, where payment, audit, moderation, and governance workflows act on the scores they return. An inference proof alone does not show whether a score belongs to the intended request, model checkpoint, numerical configuration, and verifier: in a controlled comparison that isolates the accepted statement from prover scale, a generic inference proof wrapped for on-chain checking remains valid after six classes of such service-field substitution. We formulate verifiable reward-model-as-a-service (VRMaaS) as a service-output integrity problem and present a framework in which each returned reward becomes a proof-carrying artifact. Its off-chain acceptance rule binds request identity, committed weights, verifier and configuration identifiers, proof-task order, and the digest of the exported package, so auditors can check a reward without rerunning the private RM or exposing inputs, checkpoints, or hidden activations; an optional on-chain zero-knowledge wrapper gives contracts a partial attestation of selected consistency checks and supplied service fields. The framework decomposes Transformer RM serving into arithmetic and lookup-backed proof tasks, aggregates them into a composite proof, and links hidden operands to the committed weights. A GPU prototype on 1B, 3B, and 8B reward models over 128-1024 tokens preserves 93.5-99% pairwise agreement with the baseline RM and rejects every tested service-field substitution in complete artifacts. Using exact integer witnesses and request-level aggregation of weight-binding openings, it generates complete artifacts in 49.8-206.1 seconds across the evaluated settings; auditors verify retained artifacts in 1.94-5.11 seconds, and partial public attestation costs about 217K gas. These costs suit delayed audit, settlement, and governance workflows.
查看此版本解读 →查看此版本原文 ↗