Xi Zheng, Yinghui Huang, Xiangyu Chang, Ruoxi Jia, Yong Tan
Modern AI model markets assemble models from heterogeneous pipeline contributions, including raw data, derivative data, and data-related services, whose value can depend on upstream inputs and inherited model states. Conventional valuation approaches focus on data contributions without explicitly accounting for these production relationships. They can therefore misattribute value by evaluating contributions outside their upstream production context. We formalize this pipeline-context valuation mismatch and develop Asymmetric Data Shapley (ADS), a unified framework for structure-aware value attribution. ADS translates documented production relationships into ordered valuation groups and evaluates contributions among peers conditional on their upstream production context. We provide an axiomatic characterization of the resulting allocation and quantify allocation uncertainty when evidence supports multiple valuation structures. To support valuation at larger scales or with costly utility evaluations, we develop a Monte Carlo estimator (MC-ADS) and a surrogate-assisted estimator (SA-MC-ADS). Experiments show that ignoring production dependencies systematically shifts value toward redundant derivative data and away from useful upstream data and services. Moreover, experiments on derivative data selection and on dataset procurement during multi-stage large language model fine-tuning show that decisions guided by ADS improve downstream model performance relative to the respective benchmarks.
Yangshen Deng, Xinjing Huang, Yulong Wang, Yunzhuo Liu, Jun Hu, Shuyong Tan, Chun-Chih Tseng, Mingji Han 等 11 位作者
We present HyScale, a long-context training system for learned sparse attention. Learned sparse attention is a promising architecture for million-token contexts, widely adopted in production since DeepSeek-V3.2. Selection, however, grows into the training bottleneck. For each query, it scores every previous token whereas sparse attention reads only $k$; worse, current systems materialize a full score table and repeat selection during recomputation. Our key insight is that the output of selection is not an activation, but rather a set of indices. Building on it, HyScale introduces HyTopK and selection replay. HyTopK finds each query's exact top-$k$ on the fly without a score table, while selection replay reuses forward-pass selection in recomputation. Together with holistic system optimizations tackling end-to-end training challenges, HyScale speeds up GLM-5.1 training on 1M-token sequences by 2.07$\times$ over Megatron-LM on 512 H20 GPUs. It also speeds up the CSA layers of DeepSeek-V4-Flash and the QSA layers of Qwen3.8-Flash-Next by 2.01$\times$ and 1.57$\times$ at 1M tokens on 32 H800 GPUs.
Haiyong Tang, Hanzhou Wu, Xinpeng Zhang, Athanasios V. Vasilakos
Large language models (LLMs) are valuable digital assets, yet their weights can be easily modified after unauthorized acquisition, making model provenance difficult to verify. Model fingerprinting provides a promising solution for model ownership protection without modifying the model or embedding ownership information. However, parameter-based fingerprints can be sensitive to model transformations, limiting their robustness, while representation-based fingerprints may assign high similarity to unrelated models, limiting their uniqueness. To address these limitations, we propose TrFP, an intrinsic fingerprinting method based on computational trajectories. Instead of comparing model parameters or representations, TrFP captures how representations evolve across network depth and constructs compact fingerprints using permutation-invariant trajectory descriptors. Experiments across multiple LLM families show that TrFP reliably distinguishes related and unrelated models. It also remains robust under fine-tuning, pruning, quantization, model merging, parameter permutation, and adaptive attacks, while requiring less than 0.3 MiB of fingerprint storage. Overall, TrFP provides stronger provenance discrimination than several existing state-of-the-art methods, demonstrating the effectiveness of computational trajectories for reliable LLM provenance verification.
Xiaoao Zhu, Jie Ren, Zhiqiang Li Shaanxi Normal University, Jie Zheng Northwest University Xi'an, Zhanyong Tang Northwest University Xi'an, Zheng Wang University of Leeds
Lifting stripped and highly optimized binaries to the canonical compiler intermediate representation (IR) enables program analysis when source code is unavailable. However, compiler optimizations severely distort control-flow and data-flow structure, making existing rule-based and LLM-based decompilation approaches brittle. We present BRIDGE, a system that reliably lifts optimized binaries to analysis-friendly compiler IR. BRIDGE combines control-flow-aware retrieval-augmented generation with feedback-driven verification. It uses pseudo-probe instrumentation to align optimized binary fragments with normalized IR semantics, and then employs an iterative refinement loop guided by static analysis and runtime feedback to improve executability and semantic consistency. We evaluate BRIDGE on HumanEval-Decompile and MBPP, lifting x86-64 and ARM64 binaries to LLVM IR. BRIDGE outperforms seven baselines, achieving an average of over 30% higher re-executability than the strongest general-purpose LLM baseline.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表