跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+197新增论文
+151其中 AI 相关
+16已有论文修订
+91新增 AI 导读

arXiv 最近完成扫描:2026-11-22 06:41 · 全站最近入库:2026-11-21 12:19各信源进度 →

统计口径与数据新鲜度

统计 2026-11-21 08:26 — 2026-11-22 08:26(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-22 06:41最近发现新增版本:2026-11-21 12:19库内论文最新更新时间:2026-11-20 02:59

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

29,158 已收录论文24,181 AI 相关8,992 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 5 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-22 06:39(北京时间)
版本更新 · v2arXiv:2610.01054

用任务算子刻画上下文学习动态

Capturing In-Context Learning Dynamics with Task Operators

Guangzhi Xiong, Zhenghao He, Bohan Liu, Sanchit Sinha, Wenqian Ye, Aidong Zhang

AI 速读 基于摘要

作者提出任务算子(Task Operator),把上下文学习(ICL)中注意力头的仿射变换参数提取为对注意力输出投影的解析更新,从而在无需完整示例上下文的情况下近似 ICL 效果。

cs.CL · cs.AI · cs.LG

查看原文摘要

In-context learning (ICL) enables language models to perform new tasks from demonstrations without weight updates. However, every ICL inference requires processing the full set of examples, resulting in inefficient deployments, and how ICL works mechanistically is not fully understood. Prior work compresses ICL into fixed activation vectors extracted from specific layers or positions, but these input-independent interventions fail on complex tasks where the output depends on fine-grained interactions with the input. By analyzing the ICL forward pass, we show that each attention head's output is an affine transformation of its context-masked counterpart, and that the parameters of this transformation are empirically stable across samples for a given task. Building on this, we introduce Task Operator (TO), which replays this transformation as an analytically derived update to the attention output projection. Across lexical, algorithmic, and reasoning tasks, TO achieves the best overall performance among prior methods and substantially narrows the gap between zero-shot inference and ICL. We further show that the extracted knowledge concentrates in a task-specific sparse circuit across layers and positions, and that averaging operators from disjoint demonstration batches enables effective many-shot scaling without expanding the context window. Our code is available at https://github.com/gzxiong/task_operator.

版本更新 · v5arXiv:2602.09009

ANCRe: Adaptive Neural Connection Reassignment for Efficient Depth Scaling

Yilang Zhang, Bingcong Li, Niao He, Georgios B. Giannakis

原文摘要

Scaling network depth has been a central driver behind the success of modern foundation models, yet recent investigations suggest that deep layers are often underutilized. This paper revisits the default mechanism for deepening neural networks, namely residual connections, from an optimization perspective. Rigorous analysis proves that the layout of residual connections can fundamentally shape convergence behavior, and even induces an exponential gap in convergence rates. Prompted by this insight, we introduce adaptive neural connection reassignment (ANCRe), a principled and lightweight framework that parameterizes and learns residual connectivities from the data. ANCRe adaptively reassigns residual connections with negligible computational and memory overhead ($<1\%$), while enabling more effective utilization of network depth. Extensive numerical tests across pre-training of large language models, diffusion models, and deep ResNets demonstrate consistently accelerated convergence, boosted performance, and enhanced depth efficiency over conventional residual connections.

cs.LG · cs.AI

新论文 · v1arXiv:2611.15155

PRELUDE: Separating Common Components from Target-Specific Dependencies for Multivariate Time Series Forecasting

Dongqing Song, Jianyu Xu, Haobin Ding, Xinyi Guo, Jiahao Du, Ruiqi He, Ruiqi Liu, Yuntong Xie 等 9 位作者

原文摘要

Multivariate time series forecasting often improves the prediction of a target by modeling the interactions among variables, and cross-variable modules such as cross-sectional pooling, inter-variable attention, and graph structures are used to construct such interactions. However, by intervening on the cross-variable reading of existing models at inference time, we find that although these models substantially reduce forecasting error through cross-variable interaction, their reading is nearly identical for all targets: the gain appears to come from dependencies among variables but mainly comes from the common component. We term this phenomenon spurious dependency and attribute it to common-component confounding: present in every variable, the common component affects how reading distinguishes sources, so that reading tends to be shared across targets and concentrates on a few similar averages; dependencies thus make up a small share of the read-in content, and a common component composed of several drivers is only partly captured. Theoretical analysis further shows that the common component and dependencies place different requirements on reading and should therefore be handled in different ways. Accordingly, we propose PRELUDE, a separated-reading framework that estimates the common component holistically through cross-sectional pooling and multiple common factors, and reads dependencies for each target via lead-lag relations in the innovation space obtained by prewhitening. Extensive experiments on 16 real-world datasets from four domains show that PRELUDE effectively mitigates spurious dependency and achieves the best overall performance: it attains the best MSE on 23 of the 24 short-horizon and air-quality settings and reduces the average MSE by 6.1% relative to the strongest baseline.

cs.LG

新论文 · v1arXiv:2611.01753

ReCoG: Relative Node Contexts for Multi-Domain Graph Pre-training

Yupeng Tong, Lianze Shan, Jitao Zhao, Dongxiao He

原文摘要

The success of foundation models in language and vision has motivated Graph Foundation Models (GFMs), which learn transferable knowledge through multi-domain graph pre-training. Existing methods advance GFMs through feature unification and structural learning, and in doing so they rely on the raw graph structure to organize node contexts. However, we observe that removing a large proportion of edges from source graphs during pre-training improves cross-domain transfer accuracy across four representative methods. Our theoretical analysis identifies the raw graph structure as a source of conflict in cross-graph model sharing. We therefore advocate defining node contexts using criteria consistent across graphs. Achieving this requires (1) constructing node contexts consistently across graphs without a shared positional structure, such as the token order of text or the pixel grid of images, and (2) retaining the structural and neighborhood information of the original graphs. To this end, we introduce Relative Contexts for Graphs (ReCoG) for multi-domain graph pre-training. Specifically, we first construct attribute contexts to provide a comparison rule independent of the raw graph structure. Furthermore, we construct structural contexts based on common neighbors and semantic contexts based on neighborhood attribute summaries to retain the topological and neighborhood information of the original graphs. Since all graphs share these construction rules, we pre-train a single graph encoder by reconstructing node attributes over these contexts, without domain-specific modules. We conduct extensive experiments on 18 target datasets, which demonstrate the effectiveness of ReCoG. In particular, ReCoG exceeds the strongest baseline by 7.61 percentage points in mean 1-shot cross-domain node classification accuracy and by 4.47 points in cross-type transfer to heterophilous graphs.

cs.LG

版本更新 · v2arXiv:2604.09512

Integrated electro-optic attention nonlinearities for transformers

Luis Mickeler, Kai Lion, Alfonso Nardi, Jost Kellner, Pierre Didier, Bhavin J. Shastri, Niao He, Rachel Grange

原文摘要

Transformers achieve state-of-the-art performance in language processing and computer vision. Their computation increasingly depends on hardware that can keep pace not only with large matrix multiplications, but also with nonlinear operations like Softmax within the core attention mechanism. On digital accelerators, Softmax accounts for only a small fraction of total operations, yet it can disproportionately impact overall inference latency. Here, we use thin-film lithium niobate Mach-Zehnder modulators (MZM) as analog computing elements for low-latency Transformer attention. Our high-speed nonlinearities based on the intrinsic sinusoidal MZM response achieve performance close to digital baselines in Vision Transformers and GPT-2, even under 4-bit input and output quantization. We experimentally characterize the electro-optic response at symbol rates up to 10 GBaud and assess model robustness under corresponding noise conditions. Accounting for the full signal chain, our analysis indicates more than an order of magnitude lower latency relative to previous custom electronic and photonic Softmax implementations. These results establish analog electro-optic devices as nonlinear function units well suited for integration into emerging co-packaged electronic-photonic hardware, offering a path toward high-speed Transformer inference.

cs.LG