跳到正文
研踪PaperTrail

YOUR RESEARCH, IN SIGHT

让下一篇重要论文,不再错过。

覆盖计算机全学科,以 AI 为快捷入口。关注方向、作者和关键词,把发现变成持续的研究积累。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+2新增论文
+2其中 AI 相关
+0已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描:2026-11-13 00:53 · 全站最近入库:2026-11-12 06:37各信源进度 →

统计口径与数据新鲜度

统计 2026-11-12 00:59 — 2026-11-13 00:59(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-13 00:53最近发现新增版本:2026-11-12 06:37库内论文最新更新时间:2026-11-11 08:00

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

21,675 已收录论文18,323 AI 相关6,056 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 29 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-13 00:50(北京时间)
新论文 · v1arXiv:2611.14451

一种用于实时皮肤渲染与数字化的轻量级可微方法

A Lightweight Differentiable Method for Real-Time Skin Rendering and Digitization

Mario Marquez-Balduque, Carlos Rodriguez-Pardo, Jorge Lopez-Moreno

AI 速读 基于摘要

提出可微的屏幕空间次表面散射框架,从单光单视角轻量采集直接估计适配实时渲染管线的皮肤散射参数。

cs.CV · cs.GR

查看原文摘要

Inverse rendering for skin typically relies on offline path tracing to estimate physical properties, which fail to map directly to the approximate shaders utilized in interactive applications, such as game engines. We bridge this gap by directly estimating subsurface scattering parameters tailored for real-time rendering pipelines. We introduce a differentiable Screen-Space Subsurface Scattering (SSSS) framework that enables the joint optimization of spatially varying base textures and scattering distances. To account for local variations in skin thickness, we estimate these parameters across distinct facial regions, generating a blended profile map ready for production runtime engines. Our pipeline utilizes a low-cost, single-light, single-view capture setup designed for accessible and practical appearance acquisition. Optimization is driven by a masked loss function targeting regions where subsurface scattering effects are dominant. We validate our approach on a real-world dataset of $45$ individuals across the Monk Skin Tone Scale, ensuring demographic generalization. Furthermore, we complement this evaluation with synthetic data, a multi-light setup validating our single-light assumption, and a tilted-view configuration confirming angular consistency. Results demonstrate that our method effectively recovers scattering parameters from lightweight inputs, yielding realistic skin appearance directly compatible with standard real-time shaders.

新论文 · v1arXiv:2611.12884

引导数据分配以实现稳健的子群体泛化

Guiding Data Allocation for Robust Subpopulation Generalization

Zhaoying Pan, Yipei Wang, Shenyu Lu, Xiaoqian Wang

AI 速读 基于摘要

作者通过扩展高斯特征模型框架,从理论上刻画不同训练子群体分布下的性能景观,并提出基于梯度的子群体比例调整方向,以更高效地提升调整准确率和最差群体准确率。

cs.LG · cs.CV

查看原文摘要

Machine learning models often achieve high average accuracy in the training data while performing poorly on test data due to subpopulation shift. A common data-centric solution is to enforce balanced subgroup proportions, but full balance can be costly, infeasible, and not always necessary. In this work, we study whether balanced training data is the unique optimal configuration for robust subpopulation generalization. We extend a Gaussian feature-model framework to theoretically characterize the performance landscape across general training subgroup distributions. We show that multiple configurations, including substantially imbalanced ones, can perform comparably to the balanced one. We further derive gradient-based directions for adjusting subgroup proportions to improve adjusted accuracy and worst-group accuracy more efficiently. These findings suggest that strategically allocating data offers a more flexible and principled path to robust performance than simple balancing. The code is available at https://github.com/zhaoyingpan/subpop-data-allocation.

新论文 · v1arXiv:2611.12880

Kolmogorov-Chapman映射:无轨迹学习用于模拟分子动力学

Kolmogorov-Chapman Maps: Trajectory-Free Learning for Simulating Molecular Dynamics

Majdi Hassan, Marta Skreta, Dominique Beaini, Kirill Neklyudov

AI 速读 基于摘要

提出KC-Maps,仅用无轨迹边缘样本和已知漂移扩散系数学习SDE转移核,在肽分子动力学模拟中作者报告其GPU小时数优于经典方法。

cs.LG

查看原文摘要

Stochastic differential equations (SDEs) describe stochastic processes across a wide range of scientific domains, including physics, biology, and finance. Although individual numerical integration steps are inexpensive, observing meaningful events or transitions can require millions of sequential steps, resulting in substantial wall-clock simulation time and poor hardware utilization. Deep learning offers an alternative by amortizing many integration steps into a single neural network forward pass, potentially enabling much faster simulation. However, existing supervised approaches typically require long simulated trajectory data, which restricts their applicability. In this work, we introduce Kolmogorov-Chapman Maps (KC-Maps), a method for learning the transition kernel of an SDE that relies only on the trajectory-free marginal samples together with the known drift and diffusion coefficients. Given an initial time and state of a process, KC-Maps models its distribution integrated to any given time, corresponding to the weak solution of the SDE. To recover the correct dynamics, KC-Maps satisfies two key properties: (i) it satisfies the boundary conditions, which we incorporate directly into the neural network architecture, and (ii) it respects the Kolmogorov--Chapman equation, which provides an efficient training objective. We empirically demonstrate that KC-Maps learns to integrate the dynamics of peptide molecular systems and outperforms classical molecular dynamics in terms of GPU-hours.

版本更新 · v3arXiv:2512.01067

更差的力可带来更好的结构:机器学习原子间势的轨迹级微调

Worse Forces Can Lead to Better Structures: Trajectory-Level Fine-Tuning of Machine Learning Interatomic Potentials

Evan Dramko, Yizhi Zhu, Aleksandar Krivokapic, Geoffroy Hautier, Thomas Reps, Christopher Jermaine, Anastasios Kyrillidis

AI 速读 基于摘要

作者报告通过对弛豫轨迹反向传播并直接优化最终结构,可在力预测误差上升的同时显著降低结构误差,说明力精度与下游弛豫精度可能朝相反方向发展。

cond-mat.mtrl-sci · cs.AI · cs.LG

查看原文摘要

Machine learning interatomic potentials (MLIPs) are commonly trained and evaluated by their ability to reproduce first-principles energies and forces. Practical value, however, is often determined by downstream processes such as structural relaxation. Much of MLIP literature is motivated by the expectation that improving force accuracy will produce correspondingly better downstream behavior. Although this expectation is natural, it may not hold in practice due to model inaccuracies. Here, we demonstrate that force accuracy and relaxation accuracy can be driven in opposite directions. Starting from pretrained MLIPs, we differentiate through relaxation trajectories and optimize the the final structures. This intervention produces increases in force-prediction errors while substantially reducing structural errors. Importantly, this represents a trade-off between two aspects of physical fidelity: the learned forces become less faithful to first-principles forces, while the resulting relaxed structures become more faithful to first-principles predictions. Rather than observing a discrepancy between local quality and downstream performance, we demonstrate that this discrepancy can be induced through optimization. Our findings suggest that MLIPs need not be optimized as approximations of first-principles forces, but can instead learn task-specific vector fields that guide relaxation trajectories toward more accurate final structures. By propagating supervision through complete relaxation trajectories, our framework captures dependencies between successive predictions that are absent when structure--force pairs are treated independently. More broadly, these results demonstrate that conventional force metrics can provide an incomplete assessment of downstream performance and highlights the importance of evaluating scientific machine-learning models within computational workflows.

版本更新 · v2arXiv:2611.00092

OCRia-1 技术报告

OCRia-1 Technical Report

Aman Rangapur, Antonio Jimeno Yepes, Daniel Bustamante, Nutchapol Dendumrongsup, Sebastián G. Acosta, Pierre-Loïc Doulcet, Simon Suo

AI 速读 基于摘要

OCRia-1 是一个 4B 参数的文档解析视觉语言模型,作者报告其在表格结构、图表数据与文本格式保真度上优于所评估的更大模型,并强调数据质量与奖励设计。

cs.CV

查看原文摘要

We introduce OCRia-1, a 4B vision-language model (VLM) for document parsing that excels at preserving table structure, extracting chart data and retaining text formatting. OCRia-1 handles content that has traditionally been challenging in document parsing: merged cells and spanning headers, tables with thousands of cells, chart data, and meaningful text formatting such as bold, italics, subscripts and superscripts, preserving details needed for business-critical document analysis. At the time of this report, it achieves the highest scores on tables, charts and text formatting among the systems evaluated, including a 35B model and Mistral OCR 4.1. Its mean ParseBench score over the four dimensions we evaluate is 0.87, against 0.75 for the next-best system. We emphasize data quality and reward design in training OCRia-1. Synthetic pages provide exact ground truth for long tables and rich formatting, while real pages are transcribed using a pipeline grounded in the PDF text layer. We attribute OCRia-1's performance primarily to careful data curation that prioritizes quality and diversity over quantity. Dimension-specific rewards evaluate table reconstruction, chart-data extraction and text-formatting fidelity. We train the model on the curated data using supervised fine-tuning (SFT), then refine it through reinforcement learning with verifiable rewards (RLVR). To our knowledge, OCRia-1 is the fastest published document-parsing VLM in its class, decoding at about 4,700 tokens per second with a single request on one B200 GPU, about nine times faster than the base model on stock vLLM. This is achieved through a block-diffusion speculator, quantization and custom kernels.

新论文 · v1arXiv:2611.08431

在伦理短文写作中为生成式AI使用搭建支架:一门入门计算课程三年经验报告

Scaffolding GenAI Use in Ethical Essay Writing: A Three-Year Experience Report in Introductory Computing

Asma Shakil, Paul Denny, Jun Seo, Gauravkumar Hirani, Sean Longyu Ma, Claus Brabrand

AI 速读 基于摘要

该经验报告基于三年六次调查与成绩关联分析,探讨在一年级计算机系统课程的伦理短文作业中,如何通过支架式负责任使用流程引导学生使用生成式AI。

cs.CY

查看原文摘要

As society, ethics, and professional practice become increasingly integrated across the computer science curriculum, technical courses need practical ways to support ethical reasoning. Ethical essay assignments offer one approach, but little is known about how students use GenAI when writing such essays. This experience report examines GenAI use across three years of ethical essay assignments in a large first-year computer systems course. Across six post-essay surveys from 2024 to 2026, we analyzed 1,894 responses, linked survey data with essay grades in 2025 and 2026, and introduced a scaffolded responsible-use workflow in 2026. Under voluntary-use policies, AI adoption remained steady at around two-thirds of students, rising above 90% when use was scaffolded. Students reported mainly using AI for help with grammar and style, while tool use diversified beyond ChatGPT in 2026. Grade-linked analyses showed that AI use did not independently predict essay quality, while marker evaluations indicated high levels of critical engagement when use was scaffolded. Overall, GenAI's pedagogical value appears to depend on how assessment design guides students to evaluate and critically engage with AI-generated suggestions.

新论文 · v1arXiv:2611.08298

超越量子消息:迈向量子原生网络功能

Beyond Quantum Messages: Toward Quantum-Native Network Functioning

Angela Sara Cacciapuoti, Jessica Illiano, Caterina De Risi, Marcello Caleffi

AI 速读 基于摘要

这篇 Perspective 主张量子现象不仅可作为网络传输的资源,还可直接参与网络功能的实现,并以纠缠生成与操作连通性为例勾勒量子原生功能设计空间。

quant-ph · cs.NI

查看原文摘要

The Quantum Internet is commonly envisioned as a network that transports quantum states and distributes entanglement as the key resource underpinning distributed quantum applications. This view is necessary but incomplete if quantum phenomena are confined to what the network carries or manages, while network functionalities remain grounded predominantly in classical abstractions. This Perspective advances a broader thesis: quantum phenomena can also participate directly in the realization of network functionalities. We first establish the architectural baseline by introducing entanglement as a stateful network resource whose operational condition is non-locally distributed across remote nodes, evolves through manipulation or consumption, and degrades over time through decoherence. Its evolving state must therefore remain visible to the functionalities acting on it. We then move beyond resource visibility and use two mechanisms as proof points of a broader quantum-native functional design space: entanglement generation during propagation through coherent spatial-path superposition, and operational connectivity induced by shared entanglement that can depart from the physical graph. These examples show that paths, neighbors, or participating nodes may become operationally relevant in ways determined by quantum coherence and shared entanglement, rather than by physical connectivity alone. We identify Quantum Addressing as a transversal abstraction capable of representing such relations while complementing, rather than replacing, classical addressing and signaling. The resulting perspective broadens the Quantum Internet from transporting quantum information and distributing entanglement to exploiting quantum phenomena in network functioning, opening a coupled frontier spanning routing, forwarding, control, and functionality placement.

新论文 · v1arXiv:2611.08249

几何相关性并不充分:评估扩散对接中聚焦式后训练监督

Geometric Relevance Is Not Sufficient: Evaluating Focused Post-Training Supervision in Diffusion Docking

Michael Brown, Maryam Rahimimovassagh, Mehdi Yazdani-Jahromi, Sanjeda Sara Jennifer, Ivan Garibay

AI 速读 基于摘要

在固定预算下比较扩散对接模型的后训练策略,发现聚焦于几何相关校正的监督并未稳定优于广泛续训。

cs.LG · q-bio.BM

查看原文摘要

Diffusion-based molecular docking requires learning a corrective score field across a broad corruption distribution, yet individual pose errors can differ substantially in their structural consequences. This creates a post-training conflict between preserving broad score-field coverage and concentrating supervision on corrections that appear more relevant to docking geometry. Using DiffDock, a fixed-budget study compares broad continuation with torsional supervision focused on Cartesian consequence or conditional target consistency, and with a broad/focused mixture. Evaluation separates fixed score fidelity, generated-pose quality, best-of-10 sampling capacity, and confidence-selected Top-1 docking performance. At <5 Angstrom, broad continuation had the highest mean Top-1 success among trained policies, and focused or mixed supervision showed no consistent advantage under the tested budget. The starting model had better fixed score fidelity, whereas the evaluated broad-continuation model had higher mean Top-1 success with the same aggregate best-of-10 recovery. Focusing supervision on structurally consequential or more consistently specified corrections did not consistently improve returned-pose success over broad continuation.

新论文 · v1arXiv:2611.08219

Agent-R²:通过真实来源数据合成扩展智能体以完成现实生产力工作

Agent-R$^2$: Scaling Agents for Realistic Productivity Work via Real-Source Data Synthesis

Teng Pan, Zixuan Wang, Yuchen Yan, Qifei Wu, Yao Chen, Yingqi Qu, Ruiqing Zhang, Yongliang Shen

AI 速读 基于摘要

提出基于真实职业种子、工作流证据与真实网页文件的数据合成框架 Agent-R²,并训练 4B 智能体以提升现实生产力任务表现。

cs.CL

查看原文摘要

Large language models have made rapid progress in agentic reasoning and tool use, raising expectations for agents that can take over realistic productivity work from humans. However, current agents still struggle to finish such tasks, where success requires following long-horizon professional workflows, understanding heterogeneous file contexts, and producing usable deliverables under fine-grained requirements. A key bottleneck is the lack of scalable post-training data that reflects real work intent, authentic file environments, and deliverable-level evaluation. To close this gap, we introduce Agent-R$^2$, a $\textbf{r}$eal-source data synthesis framework that scales training data for $\textbf{r}$ealistic productivity work. Instead of relying on manually specified workflows or LLM-generated files, Agent-R$^2$ grounds synthesis throughout the pipeline in real occupational seeds, corpus-derived workflow evidence, and authentic web-sourced files. From 10k web-corpus candidates, Agent-R$^2$ distills 4k task instances that pair real multi-format attachments with fine-grained, deliverable-level rubrics. On this data, we train Agent-R$^2$-4B via supervised fine-tuning and rubric-based reinforcement learning, improving both end-to-end productivity performance and file-grounded reasoning. Trained on only 4k instances, Agent-R$^2$-4B surpasses Qwen3-235B-A22B-Thinking-2507 on both GDPval (51.0$\%$ vs. 48.0$\%$) and OfficeQA (14.8$\%$ vs. 13.8$\%$). Further analysis shows that Agent-R$^2$ mitigates shortcut behavior, where agents produce superficially plausible deliverables before sufficiently grounding, constructing, or verifying them, while promoting sustained exploration.

版本更新 · v3arXiv:2601.15738

EvoDR:基于大语言模型进化调度规则的动态柔性装配流水车间调度

EvoDR: Evolving Dispatching Rules via Large Language Model for Dynamic Flexible Assembly Flow Shop Scheduling

Junhao Qiu, Haoyang Zhuang, Fei Liu, Jianjun Liu, Qingfu Zhang

AI 速读 基于摘要

提出 EvoDR 框架,利用大语言模型生成并协同进化调度规则,以应对带多产品交付、齐套供应与机器级柔性的动态柔性装配流水车间调度问题。

cs.NE

查看原文摘要

Dynamic flexible assembly flow shop scheduling with multi-product delivery, often with kitting supply and machine-level flexibility, can be found in many applications. It is difficult to design rules manually for it. Genetic programming (GP) has been used to automatically generate dispatching rules, enabling responsive scheduling while meeting high responsiveness demands. However, GP methods are dependent on fixed terminal sets and have poor interpretability. In this work, we develop an evolving dispatching rules framework (EvoDR) that leverages the semantic understanding and generation capabilities of large language models to achieve cross-domain integration of rule design and scheduling knowledge. In our proposed framework, multi-stage assembly supply decisions are modeled as priority sorting of directed edges based on heterogeneous graphs. A dual-expert co-evolution mechanism is proposed, which allows LLM-A to generate code and LLM-S conduct scheduling analysis and reflection. Guided by improvements in hybrid evaluation, adaptive rules that fit dynamic features are continuously evolved. Experimental results show that EvoDR can achieve lower average tardiness than other automatically designed state-of-the-art approaches. It also outperforms expert-designed competitors across 480 instances over 24 distinct scenarios with varying resource configurations and disturbance levels, and EvoDR demonstrates robust adaptability across diverse operating conditions.

版本更新 · v2arXiv:2610.12532

LLM驱动音色生成与插值中的结构与表征能动性

Structural and Representational Agency in LLM-Driven Timbre Generation and Interpolation

Joseph Cameron, Alan Blackwell

AI 速读 基于摘要

在一项LLM驱动音色界面的测量研究中,作者报告用户实际能改变的声音空间与界面所报告的位置可以分离:前者由结构精确刻画,后者存在显著误差。

cs.SD · cs.HC · cs.LG · cs.MM · eess.AS

查看原文摘要

Generative creative interfaces routinely report the user's position within a space of possibilities. Such displays imply that the quantities shown were measured from the artefact; in a growing class of systems they are instead predicted by a language model describing its own output, and the two are indistinguishable on screen. Most creative domains lack an agreed measurement against which such a self-report can be checked; audio admits one. In a measurement study of a working LLM-driven timbre interface, what a user can cause proves separable from what the interface reports. The first is exactly characterisable: the reachable set is a zonotope in parameter space whose dimension increases by one per sound placed, so the space the model may act within is demonstrably the user's own. The second is inaccurate. The displayed position differs from measurement by approximately three times the width of that region, and the error has two independent sources: model-predicted coordinates, whose accuracy varies twofold between the two models evaluated, and the interpolation step, which arises from combining synthesiser parameters and is unaffected by model quality. End to end, 27 of 36 refinement instructions produced no perceptible change, and under the weaker model the facility performed no action while reporting convergence. Effective agency (what a user can cause) and epistemic agency (whether they can know it) are therefore separable and independently engineered: structurally enforced constraints survive model failure, whereas guarantees resting on the model's own report do not. The failure mode that remains is silence presented as success. We close with a short protocol for building and auditing such displays.

版本更新 · v3arXiv:2605.12827

GraphIP-Bench:窃取图神经网络有多难,我们能阻止吗?

GraphIP-Bench: How Hard Is It to Steal a Graph Neural Network, and Can We Stop It?

Kaixiang Zhao, Bolin Shen, Yuyang Dai, Shayok Chakraborty, Yushun Dong

AI 速读 基于摘要

提出统一基准 GraphIP-Bench,在同一黑盒协议下系统评测图神经网络模型窃取攻击与所有权防御,并揭示防御在提取后代理模型上的失效缺口。

cs.CR · cs.AI · cs.LG

查看原文摘要

Graph neural networks (GNNs) deployed as cloud services can be *stolen* through *model-extraction attacks*, which train a surrogate from query responses to reproduce the target's behaviour, and a growing line of ownership defenses tries to prevent or trace such theft. The title of this paper asks two questions: *how hard is it to steal a GNN?*, and *can we stop it?* Prior work cannot answer either, because experiments use inconsistent datasets, threat models, and metrics. We introduce *GraphIP-Bench*, a unified benchmark which evaluates both sides under a single black-box protocol. It integrates twelve extraction attacks, twelve defenses spanning watermarking, output-perturbation, and query-pattern-detection families, ten public graphs covering homophilic, heterophilic, and large-scale regimes, three GNN backbones, and three graph-learning tasks, and it reports fidelity, task utility, ownership verification, and computational cost on shared splits, queries, and budgets. We further add a joint attack-and-defense track which runs every attack on every defended target and measures watermark verification on the resulting surrogate, which exposes the protection that a defense retains after extraction. The empirical picture is short: stealing a GNN is easy at medium query budgets and most defenses do not change this; several watermarks verify reliably on the protected model but lose most of their verification signal on the extracted surrogate, which exposes a gap that single-model evaluations miss; and heterophilic graphs are systematically harder to steal, while a cross-architecture mismatch between target and surrogate reduces but does not prevent extraction. We release *GraphIP-Bench* with reproducible scripts and configurations, and integrate the attacks and defenses into the PyGIP library.

新论文 · v1arXiv:2611.05368

GUILD:面向模仿学习的引导式迭代数据收集

GUILD: Guided Iterative Data Collection for Imitation Learning

Shivin Dass, Kevin Nguyen, Rohan Yelandur, Junhong Xu, Roberto Martín-Martín

AI 速读 基于摘要

GUILD 提出一种引导式数据收集方法,利用视觉语言模型的语义先验提出采集任务,并用预测性数据归因评估轨迹对目标部署性能的影响,形成迭代闭环以提升模仿学习的数据效率与部署表现。

cs.RO · cs.LG

查看原文摘要

Imitation learning policies in robotics are trained on datasets collected through costly real-world teleoperation. Often, the data-collection process is unguided, relying on the teleoperator's intuition, resulting in data that contributes little to overall policy performance. Recent work has shown that curating such unguided datasets can substantially improve policy performance, but these approaches operate by filtering existing datasets after collection and therefore do not prevent the acquisition of uninformative data. In this work, we propose GUILD, a novel Guided Data Collection (GDC) method that actively reasons about which data should be collected to maximize downstream policy performance. We introduce an iterative framework that alternates between 1) task proposal and data collection and 2) data evaluation to close the loop. For task proposal, we leverage semantic priors from vision-language models to identify relevant environments, object configurations, and goals for data collection. However, since semantic relevance does not imply policy improvement, we perform data evaluation. By leveraging predictive data attribution we approximate how the collected trajectories influence policy performance on a target deployment setting. This signal is then used as feedback to guide subsequent data collection, closing the data collection loop. We evaluate our approach in simulation and real robot settings, demonstrating that guided data collection substantially improves sample efficiency and deployment performance compared to passive data collection strategies, multiple baselines and experienced roboticists. More information at https://robin-lab.cs.utexas.edu/guild2026/

新论文 · v1arXiv:2611.05272

无知几何:通过激活碎片化检测大语言模型中被抑制的知识

The Geometry of Ignorance: Detecting Suppressed Knowledge in LLMs through Activation Fragmentation

Adar Peleg, Dvir Alsheich, Tomer Ashuach, Avi Mendelson

AI 速读 基于摘要

该论文提出用概念查询与匿名化改写之间隐藏状态差异(anon-vectors)的几何一致性,来区分模型真正掌握与被抑制的知识。

cs.AI

查看原文摘要

Evaluating whether a large language model (LLM) genuinely knows a concept remains a fundamental problem, since behavioral metrics can be fooled by suppression: a model that refuses to answer may still harbor recoverable knowledge. The framework measures the geometric coherence of \textit{anon-vectors}, the differences in hidden states between a concept-specific query and its anonymized paraphrase, and shows that genuine knowledge forms a tight, low-entropy cluster while ignorance forces geometric fragmentation. Across ten instruction-tuned LLMs from three model families, all fragmentation metrics reliably separate known from unknown concepts, and post-unlearning, fragmentation geometry correlates with adversarial recovery across four attack types and five unlearning algorithms, though the direction of that relationship is shown to depend on attack methodology. Furthermore, a layer-resolved analysis reveals that different adversarial mechanisms exploit this geometry at varying depths, with high attack-specific correlations that are invisible to behavioral evaluation. Together, these results constitute a single-phase, attack-free proxy for unlearning robustness that captures aspects of parametric knowledge inaccessible to output-level metrics. A reference implementation is available at https://github.com/AdarPeleg/geometry-of-ignorance

新论文 · v1arXiv:2610.40537

DAWN:基于扩散的对齐与拧开工作流网络

DAWN: Diffusion-based Alignment and unfastening Workflow Network

Hemant Kumar, Piyush Goenka, Nikhil Chopra

AI 速读 基于摘要

提出分层机器人框架 DAWN,将真实世界拧开紧固件任务拆分为对齐插入与拧开两阶段,并用扩散策略、多模态分类器和力控模块协同完成。

cs.RO

查看原文摘要

We present DAWN (Diffusion-based Alignment and unfastening Workflow Network), a hierarchical robotic framework designed to perform real-world, unfastening tasks through multi-modal sensory integration. Autonomous robotic unfastening is crucial for improving recycling processes, enhancing safety, and reducing operational costs; yet, it remains predominantly manual due to inherent challenges in precise screwdriver engagement with fasteners, detailed force execution, and robustness to environmental variability. To address the complexity of learning long-horizon, contact-rich manipulation tasks, we subdivide the unfastening task procedure into two sequential stages: (1) alignment & insertion, and (2) unfastening. DAWN consists of three key components: AlignNet, a diffusion-based policy ensuring precise screwdriver tip engagement; LatchNet, a multi-modal classifier reliably detecting successful stage transitions; and UnfastNet, which achieves stable, force-controlled screw removal. Experimental validation on a Franka Emika robot demonstrates that DAWN achieves alignment & insertion success rates comparable to trained human operators, robustly transitions between stages, and attains an impressive success rate in unfastening across various unseen scenarios. Ablation studies further elucidate critical input modalities and effective training strategies, highlighting DAWN's scalability, robustness, and data efficiency.

版本更新 · v2arXiv:2610.27049

VitronEdit:跨图像、视频与3D场景的通用视觉编辑

VitronEdit: Universal Visual Editing across Images, Videos, and 3D Scenes

Jing Wang, Yanlin Li, Mong-Li Lee, Wynne Hsu, Shengqiong Wu, Hao Fei

AI 速读 基于摘要

VitronEdit 提出一个统一框架,用共享序列表示和渐进式多模态训练,把图像与视频编辑知识迁移到数据稀缺的3D场景编辑。

cs.CV

查看原文摘要

Recent advances in unified visual editing have begun to consolidate image and video editing within a single model, while 3D scene editing remains largely separate and constrained by scarce paired training data. Given the rich image and video editing supervision now available and the common editing semantics shared across modalities, a natural question is whether this knowledge can be transferred to data-scarce 3D editing. We present VitronEdit, a universal framework for image, video, and 3D scene editing. VitronEdit adopts a shared sequence representation and progressive multimodal training to transfer editing knowledge across modalities. We further introduce geometry-aware post-training, which leverages a pretrained geometry reconstruction model to provide depth and camera-pose supervision. Surprisingly, applying geometric supervision only to 3D samples yields the best overall performance, while image and video editing also improve despite receiving no direct geometric supervision, suggesting that geometric regularization transfers through the shared backbone. Our analysis further shows that image and video training mainly transfers editing semantics to 3D, whereas 3D supervision primarily improves multi-view consistency. The project page: https://vitron-llm.github.io/VitronEdit/.

新论文 · v1arXiv:2610.40518

IndicIPR-QA:印度知识产权法抽取式问答基准

IndicIPR-QA: A Benchmark for Extractive Question Answering on Indian IPR Laws

M. Mohaiminul Islam, Muhammad Abulaish

AI 速读 基于摘要

作者构建了覆盖印度七部知识产权法的抽取式问答基准,包含4462个问答对,并报告了BM25与微调InLegalBERT之间的显著性能差距。

cs.CL

查看原文摘要

Legal Question Answering (QA) over statutory texts is a challenging task in Natural Language Processing (NLP), especially in low-resource settings with complex legal systems. It requires answers that are precise and grounded in the law. However, to the best of current knowledge, no existing extractive QA benchmark provides exact answer spans from statutory texts across all seven Indian Intellectual Property Rights (IPR) Acts. This paper presents IndicIPR-QA1, an extractive QA benchmark for Indian IPR laws. The proposed dataset contains 4,462 QA pairs from 1,116 statutory contexts across seven Acts. Questions and answers are generated using GPT-4o and validated through nine automatic quality checks, single and multi-model LLM evaluations, and a 200-pair human audit. Section-level partitioning prevents source-context overlap across the training, validation, and test sets. Further analysis examines answer span length, lexical overlap, and multi-clause structure across the corpus and individual Acts. BM25 achieves a token-level F1 score of 20.21, while fine-tuned InLegalBERT reaches 85.02 under a unified sliding-window inference protocol. The large performance gap between BM25 (20.21 F1) and InLegalBERT (85.02 F1) shows that simple word matching is not enough to answer these legal questions. Performance also varies across the seven Acts, indicating substantial differences in answer-extraction performance.

新论文 · v1arXiv:2610.40477

我们能否依赖语言模型在边缘-云连续体中进行服务编排?

Can We Rely on Language Models to Orchestrate Services across the Edge-Cloud Continuum?

Zhenghao Wu, Ruihai Dong, Reza Farahani, Hamed Haddadi, Omer Rana, Hadi Tabatabaee Malazi

AI 速读 基于摘要

系统综述39项研究,提出五维分析框架,考察语言模型在边缘-云连续体服务编排中的职责、权限、保障与证据。

cs.DC

查看原文摘要

Service orchestration across the edge--cloud continuum increasingly uses language models (LMs) to translate intents and operational evidence into scheduling, configuration, diagnosis, and remediation decisions. Yet, useful model outputs do not by themselves establish dependable orchestration: reliability also depends on where the LM enters the decision process, what authority it is granted, how actions are constrained, and what system-level evidence supports the claimed effect. Existing surveys provide limited synthesis of the links between continuum-specific orchestration decisions, LM responsibility, operational authority, assurance controls, and supporting empirical evidence. We systematically review 39 primary studies and develop a five-dimensional analytical framework for connecting system context, orchestration decisions, LM agency, operational safeguards, and evaluation evidence. The synthesis reveals predominantly hybrid designs in which LMs support interpretation, candidate generation, and mitigation planning, while downstream components commonly retain feasibility enforcement and execution authority. Bounded closed-loop remediation is emerging, but continuum-specific constraints remain unevenly represented in decision formulations, and production-scale evidence across heterogeneous edge--cloud environments remains limited. These findings suggest that reliance on LM-enabled orchestration depends on aligning decision constraints, operational authority, safeguards, and evaluation evidence with the claimed system effect.

新论文 · v1arXiv:2610.40420

NECA:面向多时间跨度ICU恶化预测的邻居事件曲线增强

NECA: Neighbor Event Curve Augmentation for Multi-Horizon ICU Deterioration Prediction

Chenyang Wang, Jiawei Theresa Tang, Xinran Liu, Xinyu Lu, Xulei Yang

AI 速读 基于摘要

NECA 是一种与模型无关的方法:它检索早期窗口特征相似的历史患者,并将其后续器官功能障碍事件的发生时间转换为多时间跨度的累积事件曲线特征,再与原始患者表示拼接,供下游预测器使用。

cs.LG · cs.AI

查看原文摘要

Early prediction of ICU organ dysfunction events supports timely clinical monitoring and intervention, but estimating short-term organ-specific risk from early ICU records remains challenging due to irregular measurements, heterogeneous clinical variables, and the need to predict risk across multiple future horizons. Existing supervised models usually treat each patient as an isolated sample and do not explicitly use the observed event timing of similar historical patients. We propose Neighbor Event-Curve Augmentation (NECA), a model-agnostic method that retrieves training-set patients with similar early-window profiles and converts their subsequent organ dysfunction event onset times into cumulative event-curve features over 6h, 12h, 24h, and 42h. These features are concatenated with the original patient representation and used by standard downstream predictors. On PhysioNet 2012 and a 4,000-stay MIMIC-IV subset, NECA improves average PR-AUC and ROC-AUC across four organ dysfunction event tasks and seven supervised learners. The largest gain appears for neurological dysfunction events, where task-level average PR-AUC increases from 0.461 to 0.557 on PhysioNet 2012. Ablations show that clinically meaningful retrieval and horizon-specific neighbor risk drive most of the improvement, while the full cumulative curve adds task-dependent temporal context. These results support similar-patient event timing as a structured temporal risk signal for ICU organ dysfunction event prediction. The source code will be publicly released upon acceptance of this paper.

版本更新 · v2arXiv:2609.37993

BITEM 在 NTCIR-19 R2C2 任务:从智能体式 RAG 流水线信号预测置信度

BITEM at the NTCIR-19 R2C2 Task: Predicting Confidence from Agentic RAG Pipeline Signals

Julien Knafou, Luc Mottin, Alexandre Flament, Paul van Rijen, Esteban Gaillac, Patrick Ruch

AI 速读 基于摘要

BITEM 用一个智能体式检索增强生成流水线同时参加 NTCIR-19 R2C2 两个子任务,置信度由编排器根据运行留下的信号计算而非让模型自评,并据此讨论 HMR 指标的缺陷与 accHMR 替代方案。

cs.CL · cs.AI · cs.IR

查看原文摘要

The BITEM team entered both subtasks of the NTCIR-19 R2C2 task with a single agentic pipeline, in which a model searches, reads and records evidence over a movie corpus while an orchestrator holds the record and rules on what may be submitted. A claim is admitted only once an entailment cascade has checked it against the passage it cites, and an answer is released only once enough checked evidence stands behind it. Each question is run three or four times, every pass retrieving from a corpus stripped of what the earlier passes have already seen. The confidence score predicted with each answer is computed by the orchestrator from what the run leaves behind and is never asked of the model, which is offered no way to rate itself. The two retrieval runs placed 4th and 5th of 22, pooling the passes was worth 0.0709 nDCG@20, and the gain was largest on the multi-hop and post-processing-heavy questions, where the organisers rank the pooled run top of the field. Sixteen of the 25 answer runs were built on passages these two runs supplied, 12 of them submitted by other teams. HMR rewards a system whose confidence is high where it answers right and low where it answers wrong. The pipeline reached an accuracy of 0.9219, 6th of 25, while the confidence scores predicted for those answers gave an HMR of 0.4915, 13th. A few rules crafted over those same recorded signals, with no further model call and no further retrieval, raise that to an accuracy of 0.9375, 5th, and an HMR of 0.6985, 9th. Ranking on HMR alone can reward a system for answering wrongly with low confidence, so we propose accHMR, the accuracy multiplied by HMR, which reports the reward in proportion to the accuracy, and on which the revised rules would have scored 0.6549, 5th. For future work, fitting a model on the numbers the pipeline already produces, rather than writing such rules by hand, would be a real step forward.

新论文 · v1arXiv:2610.40398

基于深度学习的行为漂移建模用于持续生物特征认证

Deep Learning Based Behavioral Drift Modeling for Continuous Biometric Authentication

Suprava Saha Dibya, Minar, Abdulla Al Noman, Md. Zafir Abid Shresto, Ahmed Sayef

AI 速读 基于摘要

提出两级击键动力学认证系统:固定文本登录用TCN+对比学习+PBT,登录后自由文本用四模型集成持续监测行为漂移并触发重认证。

cs.CR · cs.HC · cs.LG

查看原文摘要

This research proposes a two-level keystroke dynamics authentication system that provides defense-in-depth security through continuous behavioral drift monitoring. Level 1 employs fixed-text password authentication using Temporal Convolutional Networks (TCN) with contrastive learning and population-based training (PBT), achieving 3.77% EER for high-security login verification. Upon successful authentication, Level 2 continuously monitors user typing during natural free-text sessions using a 4-Model Deep Learning Ensemble. If suspicious behavioral drift is detected during monitoring, the system re-authenticates the user by returning to Level 1, providing a feedback loop against session hijacking and behavioral anomalies. The system explicitly models long-term behavioral drift through the fixed-text component's PBT mechanism, which adapts to temporal variations in password typing patterns. This research leverages free-text (Buffalo dataset) and fixed-text (GREYC dataset) keystroke dynamics datasets, extracting temporal and spatial features including hold times, up-down latencies, down-down intervals, and trigram patterns. The final system achieves 96%+ accuracy for fixed-text authentication (3.77% EER) and 83.53% True Acceptance Rate (TAR) at 5% False Acceptance Rate (FAR) for free-text continuous authentication (10.05% EER), demonstrating superior performance and strong resistance to impostor attacks through continuous behavioral monitoring.

新论文 · v1arXiv:2610.40362HF 社区推荐

你的学习率调度可能并不有原则:权重归一化训练中的有效学习率调度

Your LR Schedule May Not Be Principled: Effective Learning Rate Scheduling in Weight-Normalized Training

Qifan Yu, Xinyu Ma, Xingyan Bin, Di He

AI 速读 基于摘要

论文指出权重归一化训练的收益主要来自有效学习率轨迹而非归一化本身,并提出以有效学习率为调度对象的 AIR 调度。

cs.LG

查看原文摘要

Weight-normalized (WN) training is emerging as a compelling alternative to conventional weight decay (WD), with recent studies reporting substantial pretraining loss gains. We show that these gains are driven primarily by the induced effective learning rate (ELR) trajectory rather than by normalization alone. Matching ELR trajectories largely collapses the training loss dynamics of WN and WD. Schedule design should therefore target ELR rather than the nominal learning rate (LR) alone. Building on this principle, we propose AIR (Affine Inverse-square-Root decay), which combines a theoretically derived $1/\sqrt{t}$ decay with an affine adjustment to satisfy prescribed peak and terminal LRs over a finite training horizon. We further identify late-stage ELR scheduling as a key source of its gains, because a shared late schedule attenuates the effect of earlier LR choices. Experiments on 128 B-token Mixture-of-Experts pretraining demonstrate that AIR outperforms cosine and other candidate schedules in training loss, downstream performance, and hyperparameter robustness, reducing terminal training loss by approximately 0.01 relative to cosine at matched cumulative LR. Applying the same ELR trajectory to WD reproduces the corresponding training dynamics, showing that the benefits extend beyond fixed-norm training and establishing ELR as the principled scheduling object.

新论文 · v1arXiv:2610.15750Google Research

Google Earth AI 地理空间推理智能体:能力与系统性评估

Google Earth AI Geospatial Reasoning Agent: Capabilities and a Systematic Evaluation

Noam Slonim, Aviv Slobodkin, Gil Fidel, Gal Moshitch, Mimi Sun, Lily Mihalkova, Joydeep Paul, Per Bjornsson 等 37 位作者

AI 速读 基于摘要

介绍由 Gemini 驱动的 Google Earth AI 地理空间推理智能体 GRA,并发布两个基准,系统评估其工具与智能体编排的贡献。

cs.AI

查看原文摘要

Google Earth AI is a family of geospatial AI models, together with an agent that orchestrates them. We present this agent, Google Earth AI's Geospatial Reasoning agent (GRA), a Gemini-powered research system that combines geospatial data sources, foundation models, and tools to answer questions about our planet. GRA organizes these into modular domain skills and pairs them with upfront geospatial planning to decompose complex queries into multi-stage workflows. Evaluating such systems is hard: real-world problems admit multiple valid workflows, require data discovery across large catalogs, and rely on evolving ground truth. We introduce two benchmarks and release their development splits. GeoExpertQueries poses complex analytical problems grounded in practitioner use cases and refined by geospatial experts; each is paired with a consensus golden workflow from three independent experts and scored by a human-validated, tool-agnostic autorater. GeoKTools consists of templated queries with deterministic reference solvers that systematically vary the number of required tool types. We also use two external benchmarks, CBGB and UnivEARTH; the latter we find to be saturated. To isolate the contribution of GRA's tools and its agentic orchestration, we compare four configurations, each with Gemini 3.1 Pro and 3.8 Flash: a standalone LLM, the LLM with basic tools, the LLM with GRA's skills and tools but without its upfront planning and execution guidance, and GRA itself. With at most basic tools, an advanced model recovers much of the answer on templated queries but satisfies only a small fraction of the expert workflows on open-ended problems; GRA performs well on both. The largest step comes from GRA's skills and tools; on open-ended problems, its planning and execution guidance add a further gain with Gemini 3.8 Flash, letting Flash match or exceed Pro at lower cost.

新论文 · v1arXiv:2609.20738Google Research

利用生成式UI赋能教育:定制化学习互动

Harnessing Generative UI for Education: Tailored Learning Interactives

Alisa Kovshov, Anisha Choudhury, Anna Iurchenko, Amy Keeling, Avinatan Hassidim, Ayelet Shasha Evron, Ayça Çakmakli, Diana Akrong 等 29 位作者

AI 速读 基于摘要

提出“learning interactives”框架,借助Gemini的生成式UI能力自动构建定制化互动模拟,并通过教育者初步研究与专家教师的教学评分进行评估。

cs.CY

查看原文摘要

It is well established that students learn best through active engagement, particularly via interactive experiences tied to real-world concepts. Creating such experiences, however, typically requires labor-intensive design, costly development, and continuous teacher guidance to achieve measurably effective outcomes. Recent advances in generative UI open the door for a future of practice where the creation of tailored interactive experiences is automated. However, off-the-shelf models lack optimization for pedagogical precision and learning principles, often yielding unsatisfactory results for complex pedagogical constructs. To address these limitations, we present learning interactives, a structured framework leveraging the generative UI capabilities of Gemini to construct effective customized interactive simulations. We evaluate our approach through both an initial study with educators as well as a pedagogical rating of a varied set of simulations by expert teachers.

版本更新 · v2arXiv:2603.02001Berkeley BAIR

定制化OLAP:合成面向特定工作负载的“一型一机”数据库引擎

Bespoke OLAP: Synthesizing Workload-Specific One-size-fits-one Database Engines

Johannes Wehrstein, Timo Eckmann, Matthias Jasny, Carsten Binnig

AI 速读 基于摘要

提出全自动合成流水线 Bespoke OLAP,用 LLM 代码合成加迭代性能反馈,为固定分析负载从零生成专用数据库引擎。

cs.DB

查看原文摘要

Modern OLAP engines are designed to support arbitrary analytical workloads, but this generality incurs structural overhead, including runtime schema interpretation, indirection layers, and abstraction boundaries, even in highly optimized systems. An engine specialized to a fixed workload can eliminate these costs and exploit workload-specific data structures and execution algorithms for substantially higher performance. Historically, constructing such bespoke engines has been economically impractical due to the high manual engineering effort. Recent advances in LLM-based code synthesis challenge this tradeoff by enabling automated system generation. However, naively prompting an LLM to produce a database engine does not yield a correct or efficient design, as effective synthesis requires systematic performance feedback, structured refinement, and careful management of deep architectural interdependencies. We present Bespoke OLAP, a fully autonomous synthesis pipeline for constructing high-performance database engines tightly tailored to a given workload. Our approach integrates iterative performance evaluation and automated validation to guide synthesis from storage to query execution. We demonstrate that Bespoke OLAP can generate a workload-specific engine from scratch within minutes to hours, achieving order-of-magnitude speedups over modern general-purpose systems such as DuckDB.

新论文 · v1arXiv:2603.06397Google Research

通过强化学习编译扩散实现高效、属性对齐的扇出检索

Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion

Pengcheng Jiang, Judith Yue Li, Moonkyung Ryu, R. Lily Hu, Kun Su, Zhong Yi Wan, Liam Hebert, Hao Peng 等 11 位作者

AI 速读 基于摘要

R4T 先用强化学习把集合级目标“编译”成训练信号,再训练轻量扩散检索器,实现高效的单次扇出检索。

cs.IR · cs.LG

查看原文摘要

Many modern retrieval problems are set-valued: given a broad intent, the system must return a collection of results that optimizes higher-order properties (e.g., diversity, coverage, complementarity, coherence) while remaining grounded with respect to a fixed database. Set-valued objectives are typically non-decomposable and are not captured by existing supervised (query, content) datasets which only prioritize top-1 retrieval. Consequently, fan-out retrieval is often employed to generate diverse subqueries to retrieve item sets. While reinforcement learning (RL) can optimize set-level objectives via interaction, deploying an RL-tuned LLM for fan-out retrieval is prohibitively expensive at inference time. Conversely, diffusion-based generative retrieval enables efficient single-pass fan-out in embedding space, but requires objective-aligned training targets. To address these issues, we propose R4T (Retrieve-for-Train), which uses RL once as an objective transducer in a three-step process: (i) train a fan-out LLM with composite set-level rewards, (ii) synthesize objective-consistent training pairs, and (iii) train a lightweight diffusion retriever to model the conditional distribution of set-valued outputs. Across large-scale fashion and music benchmarks consisting of curated item sets, we show that R4T improves retrieval quality relative to strong baselines while reducing query-time fan-out latency by an order of magnitude.

版本更新 · v2arXiv:2506.15924Google Research

SNPeek:面向机密虚拟机隐私应用的侧信道分析

SNPeek: Side-Channel Analysis for Privacy Applications on Confidential VMs

Ruiyi Zhang, Albert Cheu, Adria Gascon, Daniel Moghimi, Phillipp Schoppmann, Michael Schwarz, Octavian Suciu

AI 速读 基于摘要

作者提出开源工具包 SNPeek,在生产级 AMD SEV-SNP 硬件上提供可配置的侧信道追踪原语,并结合统计与机器学习分析流程,自动估计隐私应用在机密虚拟机中的泄漏。

cs.CR

查看原文摘要

Confidential virtual machines (CVMs) based on trusted execution environments (TEEs) enable new privacy-preserving solutions. Yet, they leave side-channel leakage outside their threat model, shifting the responsibility of mitigating such attacks to developers. However, mitigations are either not generic or too slow for practical use, and developers currently lack a systematic, efficient way to measure and compare leakage across real-world deployments. In this paper, we present SNPeek, an open-source toolkit that offers configurable side-channel tracing primitives on production AMD SEV-SNP hardware and couples them with statistical and machine-learning-based analysis pipelines for automated leakage estimation. We apply SNPeek to three representative workloads that are deployed on CVMs to enhance user privacy-private information retrieval, private heavy hitters, and Wasm user-defined functions-and uncover previously unnoticed leaks, including a covert channel that exfiltrates data at 497 kbit/s. The results show that SNPeek pinpoints vulnerabilities and guides low-overhead mitigations based on oblivious memory and differential privacy, giving practitioners a practical path to deploy CVMs with meaningful confidentiality guarantees.

版本更新 · v3arXiv:2406.01388Google Research

AutoStudio:在多轮交互式图像生成中塑造一致的主体

AutoStudio: Crafting Consistent Subjects in Multi-turn Interactive Image Generation

Junhao Cheng, Xi Lu, Hanhui Li, Khun Loun Zai, Baiqiao Yin, Yuhao Cheng, Yiqiang Yan, Xiaodan Liang

AI 速读 基于摘要

提出免训练的多智能体框架 AutoStudio,通过 LLM 智能体与改进的扩散模型,在多轮交互中保持多主体一致性。

cs.CV

查看原文摘要

As cutting-edge Text-to-Image (T2I) generation models already excel at producing remarkable single images, an even more challenging task, i.e., multi-turn interactive image generation begins to attract the attention of related research communities. This task requires models to interact with users over multiple turns to generate a coherent sequence of images. However, since users may switch subjects frequently, current efforts struggle to maintain subject consistency while generating diverse images. To address this issue, we introduce a training-free multi-agent framework called AutoStudio. AutoStudio employs three agents based on large language models (LLMs) to handle interactions, along with a stable diffusion (SD) based agent for generating high-quality images. Specifically, AutoStudio consists of (i) a subject manager to interpret interaction dialogues and manage the context of each subject, (ii) a layout generator to generate fine-grained bounding boxes to control subject locations, (iii) a supervisor to provide suggestions for layout refinements, and (iv) a drawer to complete image generation. Furthermore, we introduce a Parallel-UNet to replace the original UNet in the drawer, which employs two parallel cross-attention modules for exploiting subject-aware features. We also introduce a subject-initialized generation method to better preserve small subjects. Our AutoStudio hereby can generate a sequence of multi-subject images interactively and consistently. Extensive experiments on the public CMIGBench benchmark and human evaluations show that AutoStudio maintains multi-subject consistency across multiple turns well, and it also raises the state-of-the-art performance by 13.65% in average Frechet Inception Distance and 2.83% in average character-character similarity.

版本更新 · v2arXiv:2305.14788Berkeley BAIR

让语言模型适应压缩上下文

Adapting Language Models to Compress Contexts

Alexis Chevalier, Alexander Wettig, Anirudh Ajith, Danqi Chen

AI 速读 基于摘要

将预训练语言模型改造为 AutoCompressors,用无监督学习把长上下文压缩成摘要向量作为软提示,以扩展上下文窗口并加速推理。

cs.CL

查看原文摘要

Transformer-based language models (LMs) are powerful and widely-applicable tools, but their usefulness is constrained by a finite context window and the expensive computational cost of processing long text documents. We propose to adapt pre-trained LMs into AutoCompressors. These language models are capable of compressing long contexts into compact summary vectors, which are then accessible to the model as soft prompts. Summary vectors are trained with an unsupervised objective, whereby long documents are processed in segments, and summary vectors from all previous segments are used in language modeling. We fine-tune OPT and Llama-2 models on sequences of up to 30,720 tokens and show that AutoCompressors can utilize long contexts to improve perplexity. We evaluate AutoCompressors on in-context learning by compressing task demonstrations and find that summary vectors are good substitutes for plain-text demonstrations, increasing accuracy while reducing inference costs. Finally, we explore the benefits of pre-computing summary vectors for large corpora by applying summary vectors to retrievalaugmented language modeling and a passage re-ranking task. Overall, AutoCompressors emerge as a simple and inexpensive solution to extend the context window of LMs while speeding up inference over long contexts.