跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1199新增论文
+950其中 AI 相关
+132已有论文修订
+97新增 AI 导读

arXiv 最近完成扫描:2026-11-20 08:48 · 全站最近入库:2026-11-20 16:53各信源进度 →

统计口径与数据新鲜度

统计 2026-11-19 18:59 — 2026-11-20 18:59(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-20 08:48最近发现新增版本:2026-11-20 16:53库内论文最新更新时间:2026-11-20 02:59

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

28,609 已收录论文23,738 AI 相关8,785 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 253 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-20 08:46(北京时间)
版本更新 · v3arXiv:2608.04457

Eigenius:一个具有认知分层与制度中介推理的类型化知识图谱数据库管理系统

Eigenius: A Typed Knowledge-Graph DBMS with Epistemic Stratification and Institution-Mediated Reasoning

Hans-Martin Will, Allen L. Brown, Matthew Fuchs

AI 速读 基于摘要

Eigenius 是一个开源类型化知识图谱 DBMS,通过将类型系统、存储引擎与集成协议耦合于统一内核,使数据溯源成为结构性不变量,并在 Lean 4 中机械化其元理论。

cs.DB · cs.AI · cs.LO

查看原文摘要

As "AI Scientists" emerge to drive research via the Model Context Protocol (MCP), systems relying on ephemeral scripts will fail. The sheer scale of stateful, interconnected evidence requires a machine-walkable warranty grounded in a purpose-built database architecture. Eigenius is an open-source, typed knowledge-graph DBMS built on a single premise: answering the audit question ("what do you know, and what is your warranty?") requires a unified kernel. By tightly coupling the type system, storage engine, and integration protocol, Eigenius turns data provenance into a structural invariant rather than a property reconstructed across subsystem boundaries. The kernel rests on three pillars whose metatheory is mechanized in Lean 4: a dependent type theory (EigenTT) separating confluent conversion from ontology subsumption and keeping proof irrelevance axiomatic so distinct warrants remain inspectable; proof-calculus institutions acting as strongly typed integration boundaries; and a content-addressed immutable storage layer. Epistemic status (declared/observed/derived/verified) forms a bounded lattice over a justification logic enforced at commit. Cross-system comorphisms carry explicit axiom policies (Adopt, Discharge, Relativize), transport witnesses at commit, and materialize outputs into the graph, while EigenQL provides a Grothendieck query view across all hosted logics and shared on-chain IRs collapse multi-system translations to identity. Pairing empirical justification logic with an in-process Lean 4 term checker, an end-to-end recomputation of a published Nature study from pinned data discharges all 52 derived conclusions and surfaces four machine-checked discrepancies in the original study.

新论文 · v1arXiv:2611.08106

清洗还是回忆?基于LLM的数据清洗中的知识依赖

Cleaning or Recalling? Knowledge Dependence in LLM-based Data Cleaning [Experiment, Analysis & Benchmark]

Jingru Yang, Honglu Zhang, Jiayi Mi, Ju Fan, Zheng Chen, Xiaoyong Du

AI 速读 基于摘要

提出CoRE协议,通过对比公开基准、新鲜真实记录与配对名称替身,检验LLM数据清洗表现是否依赖对实体名称的既有知识。

cs.DB

查看原文摘要

High scores on public data-cleaning benchmarks can reflect knowledge of familiar entities, making it difficult to assess how well an LLM repairs unfamiliar data. We present CoRE (Cleaning or Recalling), a protocol comparing public benchmarks with fresh real records and paired name surrogates built from English words or syllables. The surrogates retain error positions and non-identity values while transferring the benchmark's error mechanisms. Across 21 controlled tables and eight LLMs, repair performance varies substantially with entity names on Hospital and, for commercial models, Movies. Renaming removes some information needed for repair, such as a film's identifier mapping. It also affects repairs whose candidates remain available: replacing entity names with word-based surrogates lowers DeepSeek-V4-Pro's year accuracy from 91% to 18%. On Hospital and Movies, commercial models score higher on fresh records than on word-based surrogates. A screened continuation probe finds no positive evidence of benchmark-row memory in the tested models. Familiar facts can conflict with the task. Even with explicit table-evidence instructions, commercial models rewrite assigned directors at rates of 0.000-0.404 on known films. Reference scoring can also disagree with external records. On Rayyan, an ISSN correction matching a title-matched PubMed record receives no credit, whereas 94% of TabClean's credited repairs reproduce the reference date transformation. We provide controlled tables, scoring code and materials for offline replay.

新论文 · v1arXiv:2611.07971

ArcadeDB in Python: An In-Process Multi-Model Database

Taewoon Kim

原文摘要

Some Python workflows need transactional reads and writes over documents, traversals over a graph, and similarity search over vector embeddings. In practice these are split across separate systems (a document store, a graph engine, and a vector index), and the data is copied between them and kept consistent by application code. ArcadeDB is an open-source multi-model database engine, written in Java, that supports all three access patterns and can be embedded in a host process. We present arcadedb-embedded, a Python package that runs ArcadeDB inside the Python process via JPype, using a Java Runtime Environment bundled in the wheel. One pip install gives a database with documents, a property graph, and vector search, and needs no Java installation. Query results can be returned as Python lists, NumPy arrays, pandas DataFrames, or Arrow tables. We demonstrate a hybrid retrieval workflow that chains vector search, SQL filtering, and graph traversal over one dataset. We benchmark arcadedb-embedded against SQLite, DuckDB, LadybugDB, and Chroma, and report its costs.

cs.DB · cs.IR

版本更新 · v2arXiv:2609.03315

Lantern: Finding Committable Transactions via Back-Propagation on DAGs

Denglong Li, Gerui Wang, Tian Guan, Mingchao Wan

原文摘要

Existing concurrency control protocols either introduce nondeterminism, resulting in a serial execution-replay dependency between primary and replica nodes, or rely on impractical prior knowledge of transaction read-write sets. In this paper, we present Lantern, a deterministic concurrency control protocol tailored for high-performance transaction processing systems operating without prior knowledge. The key insight of Lantern is that all zero-out-degree transaction vertices in the local dependency graph can be safely committed in ascending order using an overwrite-permissive strategy. We further introduce a novel Back-Propagation mechanism that iteratively propagates dependency states from sink to source vertices to identify additional committable transactions. We also propose Conflict-Free Batch Selection (CFBS) for read-modify-write intensive scenarios. We integrate Lantern into the open-source blockchain platform ChainMaker. Extensive evaluations on YCSB and SmallBank benchmarks demonstrate that Lantern achieves up to a 4.2x throughput speedup over Aria and improves the throughput of ChainMaker's execution layer by at least 2.2x.

cs.DC · cs.DB · cs.PF

新论文 · v1arXiv:2611.07432

Typed Knowledge over Constructive Institutions

Hans-Martin Will

原文摘要

Type theories designed for mathematics or program verification need only that a proposition holds; proof irrelevance in conversion erases, at the kernel level, how a claim was established. A persistent knowledge store, by contrast, must retain that record: who established a claim, by which derivation or external tool, and under which assumptions. This volume presents the theory of Eigenius---a store of typed knowledge over constructive institutions---and develops its metatheory from that requirement. The kernel theory is a dependent type theory with cumulative universes, an impredicative universe of propositions, inductive families, and an intensional identity type. Proof irrelevance is an axiom with an explicit witness rather than a conversion rule, so proofs of a proposition are provably equal yet remain distinct terms. Definitional equality is joinability under an orthogonal rewriting system---confluent, injective on type formers, and decidable by a type-free test---while a separate subsumption judgment covers universe cumulativity, codomain-covariant function types, and declared class inclusions. We prove consistency via a set-theoretic model; develop a justification logic over the kernel's propositions to track evidence and support; organize environments as an institution so that store extensions are signature morphisms and external verdicts enter via comorphisms; formulate a typed knowledge-graph theory with records, classes, and refinements; and prove the correctness of conversion, normalization by evaluation, and bidirectional type checking relative to termination. A Lean 4 formalization accompanies the volume.

cs.LO · cs.AI · cs.DB

版本更新 · v2arXiv:2606.00734

EMA: Approximate Nearest Neighbor Search with General Attribute Filtering and Dynamic Updates

Mocheng Li, Xiao Yan, Baotong Lu, James Cheng, Chenhao Ma

原文摘要

Filtering approximate nearest neighbor search (Filtering ANNS) retrieves approximate nearest neighbors subject to attribute constraints in applications such as recommendation, retrieval-augmented generation (RAG), and LLM agent memory. However, predicate-specialized methods target either range or label filtering and can incur substantial index construction time, whereas predicate-agnostic methods support arbitrary predicates but struggle to sustain throughput and recall across a wide range of selectivities. In this paper, we propose EMA, a Filtering ANNS algorithm that supports multi-predicate queries over mixed numerical and categorical attributes as well as efficient dynamic updates. EMA attaches compact summaries, called Markers, to graph edges to record the attributes of candidates removed by geometric pruning. During query processing, EMA performs Marker-guided joint search with bounded edge recovery, enabling efficient filtering while preserving graph navigability. On multi-predicate workloads, EMA achieves 1.20x-28.18x the query throughput of the best competing baseline at comparable recall. In our complete-record replacement experiment, EMA retains approximately 98% of its initial query throughput without rebuilding the graph.

cs.DB

新论文 · v1arXiv:2611.07339

DiagDBench: Evaluating Agentic LLMs on Interactive Database Diagnosis and Anomaly Remediation

Xinxin Zhao, Rong Kang, Tieying Zhang, Cuiping Li, Hong Chen, Jing Zhang

原文摘要

Database diagnosis and anomaly remediation are essential to reliable database management system (DBMS) operation. Despite advances in large language model (LLM)-based database agents, constructing live DBMS environments that reproduce diverse anomalies at scale remains challenging, limiting the availability of executable samples for database agent training and evaluation. To this end, we introduce DiagDBench, a large-scale benchmark for training and evaluating database agents on interactive diagnosis, anomaly remediation, and optional workload performance optimization across diverse, realistic anomaly scenarios. Each sample pairs a user request, comprising an issue description and an anomaly-verified workload under which the issue was observed, with a self-contained, executable DBMS environment that supports real-time monitoring, interactive diagnosis, remediation, and workload execution. To construct these samples at scale, we propose an LLM-based framework with two complementary pipelines: Manual-Guided Anomaly Synthesis, which sequentially synthesizes user requests, database modifications, and anomaly-injection scripts grounded in DBA manuals and official DBMS documentation; and Incident-Guided Anomaly Reproduction, which reconstructs anomaly samples end to end from real-world enterprise online diagnosis incidents. Built using this framework with sample filtering, expert review, and refinement, DiagDBench comprises 6,940 synthesized training samples and a held-out test set of 454 expert-annotated samples, spanning three hardware configurations, two DBMSs, and three database schemas. Extensive evaluations of database agents powered by multiple LLMs demonstrate that even the best-performing agent, ReAct with GPT-6 Astra, scores only 56.24, whereas most agents score below 45, underscoring the difficulty of DiagDBench.

cs.DB

版本更新 · v2arXiv:2610.37407

Constraining the Commit Boundary: A DIO/ODI-Informed Authorization Guardrail for Agentic Entity Resolution

Daniel Beranek

原文摘要

Agentic Entity Resolution (ER) systems iteratively plan actions, retrieve evidence, revise hypotheses, and propose graph mutations. An erroneous identity hypothesis may condition later retrieval and produce apparent confirmation. This paper proposes a Digital Identity Optimization (DIO) and Ontology of Digital Identity (ODI)-informed authorization boundary between evidence available for reasoning and evidence eligible to authorize persistent identity-changing mutations. A domain-governed constraint-binding function maps selected reconstruction-failure distinctions to identity-preserving constraints. An abstract two-phase supervisor permits commitment only when claim-specific authorization-eligible evidence supports every required proposition, relevant conflicts are resolved, scope is valid, and bound state and policy versions remain current. Admission as UNVERIFIED does not confer authorization eligibility; separate claim-specific promotion is required, and insufficient support or unresolved material conflict defers the mutation. The guarantee is conditional on adequate domain policy, sound claim-specific evidence promotion, and faithful enforcement; the specification does not independently establish promotion correctness or semantic truth. The proposed contribution is a DIO/ODI-informed diagnostic-to-constraint organization integrated with an explicit authorization contract for claim-specific evidence promotion and mutation-specific proposition requirements. Scope binding, conflict handling, and commit-time version checks are treated as necessary contract elements, not as novel mechanisms in isolation. This is a conceptual architecture without implementation or empirical validation. It does not prevent erroneous hypotheses or hypothesis-conditioned retrieval; dependency-aware evidence flow remains future research.

cs.AI · cs.CR · cs.DB

版本更新 · v2arXiv:2605.24183

AvalancheBench:通过潜在世界恢复评估企业智能体驱动的语义查询处理

AvalancheBench: Evaluating Enterprise Agent-Driven Semantic Query Processing Through Latent World Recovery

Darek Kleczek, Fuheng Zhao, Alexander W. Lee, Julien Tissier, Pawel Liskowski, Ugur Cetintemel, Anupam Datta

AI 速读 基于摘要

提出多模态企业分析基准 AvalancheBench,用潜在世界规格生成观测数据,考察数据智能体能否在端到端分析中正确决定何时及如何使用语义算子,并量化错误在问题间的“雪崩式”传播。

cs.DB · cs.AI · cs.LG

查看原文摘要

As LLMs improve, data agents are emerging as a new interface for analytics, while semantic query processing engines use LLMs to operate over structured and unstructured data. Existing benchmarks largely evaluate agents and semantic query engines in isolation, leaving it unclear whether agents can decide when and how to use semantic operators in end-to-end analysis. We introduce AvalancheBench, a multimodal enterprise analytics benchmark that generates observed data from latent world specifications and evaluates data agents on their ability to recover that world. The benchmark's 3 validated tasks contain 58 questions spanning text, images, and documents. Questions are provided together as a multi-part analytical brief and may share intermediate results, exposing the avalanche effect: errors can propagate through legitimate dependencies, and inappropriate reuse can cause otherwise unrelated questions to fail. AvalancheBench measures analytical fidelity, monetary cost, and capability-adjusted token cost (CATC), a new metric that weights token use by model capability to separate technical efficiency from provider pricing. We evaluate 5 model-harness configurations across 3 agent harnesses using Snowflake Cortex AISQL as the engine, with the best configuration reaching macro analytical fidelity of 0.82. Agents fail primarily in constructing the analysis: failing to recover facts that require semantic computation costs 45.1% of lost points, yet incorrect operator outputs account for only 2.5%, versus 36.3% from agents misusing or avoiding semantic operators. Misinterpreting requirements costs the most, at 51.5%. Errors also spread between questions: 29% of lost points come from reusing another question's results. Finally, monetary cost and CATC produce different Pareto frontiers, distinguishing economically attractive configurations from those that are technically resource efficient.

版本更新 · v3arXiv:2606.27571

Kimball 数据仓库架构:评估一致性数据(Conformed Data)相对于 Inmon 模型的挑战

Kimball's Data Warehouse Architecture: Evaluating the Challenges of Conformed Data against the Inmon Model

Júlio Rocha, Salviano Soares, António Valente

AI 速读 基于摘要

该文围绕 Kimball 以一致性维度为核心的数据总线架构与 Inmon 的 hub-and-spoke 架构展开比较,重点讨论一致性数据面临的挑战。

cs.DB

查看原文摘要

In recent decades, driven by rapid data growth, organisations have faced the need to restructure their storage frameworks to efficiently handle queries requested by employees through available enterprise applications. Investigating this need involves examining the classic approaches of William H. Inmon, widely known as the father of Data Warehousing, and Ralph Kimball. Although both shared the same core concerns, Kimball later suggested an alternative architecture focused primarily on user needs. According to Ariyachandra and Watson, Inmon's "hub-and-spoke" architecture and Kimball's data bus framework featuring conformed dimensions stand out among alternative approaches. A comparison across these architectures highlights four key aspects: Information Quality, System Quality, Individual Impacts, and Organisational Impacts. Although Kimball and Inmon proposed contrasting solutions, they did not view each other as rivals. For instance, in one of the editions of the book "The Data Warehouse Toolkit", published by Kimball in 1996, the back cover features a note by Inmon stating that it is "one of the definitive books of our industry. If you take time to read only one professional book, make it this book.

版本更新 · v3arXiv:2610.20527

JetStream: Generating Verified Query Accelerators for Existing Database Systems

Akhilesh Balasingam, Amadou Ngom, Geoffrey X. Yu, Tim Kraska

原文摘要

Recent work has shown that LLMs can synthesize highly instance-optimized database systems for fixed workloads, but to date these systems only replace the DBMS's native storage and assume static data: users must migrate their data, often give up ad-hoc queries, lose acceleration once the data changes, and lack formal correctness guarantees. We present JetStream, which instead generates query-specific accelerators that extend an existing DBMS. Given a set of parameterized query templates, a staged agentic workflow synthesizes accelerators that run directly over the DBMS's native tables and may maintain custom auxiliary state, while the host DBMS remains responsible for storage, transactions, recovery, and all the other queries. A fixed, engine-neutral substrate keeps auxiliary state transactionally consistent under writes, while generated backend adapters handle engine-specific hooks. JetStream generates two kinds of accelerators: tested accelerators, which may use any optimization and are validated by differential testing, and verified accelerators, which trade some performance for a machine-checked proof that every answer satisfies a trusted specification derived from the SQL. On TPC-H at SF=20, JetStream achieves a $584\times$ geomean read speedup over DuckDB ($89\times$ verified) and retains $139\times$ ($46\times$ verified) under a continuous update stream of 10,000 modified rows per second.

cs.DB

版本更新 · v2arXiv:2512.06988

Space efficient implementation of hypergraph dualization in the D-basis algorithm

Skylar Homan, Mikhail Khlystun, Anoop Krishnadas, Kira Adaricheva

原文摘要

We present a new implementation of the $D$-basis algorithm called the Small Space which considerably reduces the algorithm's memory usage for data analysis applications. The previous implementation delivers the complete set of implications that hold on the set of attributes of an input binary table. In the new version, the only output is the frequencies of attributes that appear in the antecedents of implications from the $D$-basis, with a fixed consequent attribute. Such frequencies, rather than the implications themselves, became the primary focus in analysis of datasets where the $D$-basis has been applied over the last decade. The $D$-basis employs a hypergraph dualization algorithm, and a dualization implementation known as Reverse Search allows for the gradual computation of frequencies without the need for storing all discovered implications. We demonstrate the effectiveness of the Small Space implementation by comparing the runtimes and maximum memory usage of this new version with the current implementation.

cs.DB · cs.IR

新论文 · v1arXiv:2611.06628

From Certified Pairs to a Certified Hub: Reasoner-Checked Crosswalks for the Information Exchange Standard

Fabio Rovai

原文摘要

A national digital twin was conceived as an ecosystem of twins connected through mappings to a common foundation, but the roadmap that promises to confirm those mappings are valid never says what valid means. We give the test a precise form, coherence and conservativity of the merged ontology, borrowed from logic-based alignment repair, and apply it to the Information Exchange Standard (IES), first pair by pair and then to whole hubs. Crosswalks from IES into thirteen ontologies across seven sector groups, anchored on the building ontologies BOT, Brick, and SAREF4BLDG, pass through separate stages of matching, human review, certification of each SKOS correspondence against the two class hierarchies, and reasoning with HermiT. Pair by pair, the stages catch different failures: certification alone removed all 526 conservativity violations of the raw crosswalks, review alone left 73, and together they made eleven of twelve overlapping pairs coherent and conservative. Pairwise certification does not compose. Merging the three certified building crosswalks into one hub made five Brick classes unsatisfiable, through a homonym between the IES and Brick notions of a floor that only BOT's disjointness axioms expose; rejecting that one row gave a coherent and conservative hub that entails 290 cross-ontology links nobody wrote down. The core hub is inconsistent before any crosswalk is added, because two published standards, the Open Energy Ontology and BFO 2020, contradict each other; with either one left out it is coherent and conservative. A hub, not a pair, is the unit an ecosystem of twins has to certify.

cs.AI · cs.DB · cs.LO

版本更新 · v2arXiv:2601.22183

COL-Trees: Efficient Hierarchical Object Search in Spatial Networks

Tenindra Abeywickrama, Muhammad Aamir Cheema, Sabine Storandt

原文摘要

Many spatial network applications, such as map-based services, rely heavily on efficient methods to search for relevant nearby points-of-interest (POIs). A $k$ Nearest Neighbor ($k$NN) query is one such example that finds the $k$ closest POIs from an agent's location. While most existing techniques for spatial networks focus on retrieving nearby POIs for a single agent, these search heuristics do not translate to many other applications. For example, Aggregate $k$ Nearest Neighbor (A$k$NN) queries require POIs that are close to multiple agents. $k$ Farthest Neighbors ($k$FN) queries require POIs that are the antithesis of nearest. Such problems naturally benefit from a hierarchical approach, but existing methods rely on Euclidean-based heuristics, which have diminished effectiveness in spatial networks such as road network graphs. We propose a novel data structure, COL-Tree (Compacted Object-Landmark Tree), to address this gap by enabling efficient hierarchical graph traversal using a more accurate landmark-based heuristic. We then present query algorithms that utilize COL-Trees to efficiently answer A$k$NN, $k$FN, and other queries. Our experiments evaluate performance on real-world and synthetic datasets including multiple types of spatial networks, such as road networks of varying sizes and edge weights, terrain networks, and 2D physics meshes. The results demonstrate that our techniques significantly outperform existing approaches, achieving up to three orders of magnitude improvement. Moreover, this comes at a small pre-processing overhead in both theory and practice.

cs.DB · cs.AI · cs.DS

版本更新 · v3arXiv:2601.20482

ConStruM: A Structure-Guided LLM Framework for Context-Aware Schema Matching

Houming Chen, Zhe Zhang, H. V. Jagadish

原文摘要

Column matching is a central task in reconciling schemas for data integration. Column names and descriptions are valuable for this task. LLMs can leverage such natural-language schema metadata. However, in many datasets, correct matching requires additional evidence beyond the column itself. Because providing complete schema metadata remains impractical for most LLMs and can obscure the relevant evidence even when it fits, the core challenge becomes to select and organize the most useful contextual information. We present ConStruM, a structure-guided framework for budgeted evidence packing in schema matching. ConStruM constructs a lightweight, reusable structure in which, at query time, it assembles a small context pack emphasizing the most discriminative evidence. ConStruM is designed as an add-on: given a shortlist of candidate targets produced by an upstream matcher, it augments the matcher's final LLM prompt with structured, query-specific evidence so that the final selection is better grounded. For this purpose, we develop a context tree for budgeted multi-level context retrieval and a global similarity hypergraph for grouping and differentiating confusable columns. Across HRS-B and PSID-B, ConStruM achieves a mean combined accuracy of 95.4%, compared with 51.6% for the no-context LLM, and ranks first among all eleven evaluated methods on both benchmarks. On MIMIC--OMOP and Synthea--OMOP, attaching it to our Matchmaker reconstruction produces no statistically significant accuracy change. Together, these results show that ConStruM can deliver large gains when matching depends on context while remaining compatible with a strong matcher in the other tested settings.

cs.DB

版本更新 · v2arXiv:2611.05176

Extending the MM-AR Metamodeling Platform with Process Mining Capabilities

Geraldine Hürzeler, Hans-Georg Fill, Daniel Borcard

原文摘要

Process mining derives process models from event data recorded by information systems, but its results are usually presented as two-dimensional visualizations that are separate from the environments in which enterprise models are created and maintained. Metamodeling platforms for extended reality (XR), such as MM-AR, allow users to define modeling languages and to display models in 2D and 3D, but they do not provide access to event data or mining algorithms so far. In this paper we therefore present a first proof-of-concept for showing how an XR-based metamodeling platform can be extended with process discovery capabilities without changes to the platform core through a separate Python service based on the open source PM4Py process mining library. We demonstrate the approach using three common discovery algorithms for obtaining Petri nets and BPMN models based on CSV or XES event logs. A technical evaluation using four public event logs shows that the element counts of the created model instances match those of the PM4Py models in all 16 configurations. The integration adds between about 8 and 53 seconds per run, and this overhead grows with the size of the discovered model rather than the size of the event log. Although the prototype does not yet use the third dimension to encode process information, it established the foundations for studying such approaches.

cs.SE · cs.DB · cs.HC

新论文 · v1arXiv:2611.05986

A Comparative Study of Importance Attribution to Columns in Deep Learning over Relational Databases

Hila Levi-Yosefi, Shunit Agmon, Ester Livshits, Agapi Rissaki, Benny Kimelfeld

原文摘要

Deep learning over relational databases has led to increasingly effective prediction models, motivating a growing need to explain their behavior. We study explanation methods that assign importance scores to individual database attributes, focusing on a fundamental question: to what extent does the choice of importance-attribution method matter? We develop a methodology to systematically compare importance-attribution methods and analyze their similarities and differences. In particular, we study two aspects of the explanations: which attributes they deem important, captured by differences in attribute rankings, and how importance scores vary across attributes, captured by differences in score distributions. We further investigate whether these differences are reflected in the conventional measure of explanation quality, fidelity. We apply our methodology in an extensive empirical study of four methods: Monte Carlo Shapley, Kernel SHAP, single-attribute impact, and mask learning. We experiment with multiple datasets and tasks using two RDL architectures: heterogeneous GraphSAGE and Relational Graph Transformers. Our experiments show that explanation methods behave differently depending on the prediction type: regression or classification. In regression, most methods exhibit high agreement in attribute rankings, whereas classification yields considerably less consensus. Notably, while mask learning frequently diverges in its importance distributions, Shapley-based methods and single-attribute impact often converge to similar explanations. This suggests that, in many settings, single-attribute impact can replace Shapley at substantially lower computational cost. Overall, our study provides methodological tools for comparing importance-attribution methods, and empirical evidence that their choice has important consequences for interpreting relational deep learning models.

cs.LG · cs.DB

新论文 · v1arXiv:2611.05918

Autonomous ETL Pipelines for Salesforce M&A Consolidation: A Reference Framework, Jitterbit Mapping, and Reproducible Simulation Study

Imran Basha Shaik, Sahla Sultana, Khader Moinuddin Shaik

原文摘要

Merging companies that each run Salesforce must consolidate their customer data into a single organization while both source systems remain in use. This requires reconciling different schemas and picklists, resolving duplicate customers across and within organizations, preserving relationships between records, and keeping the target current until cutover. This article proposes a reference framework for an autonomous extract-transform-load pipeline for this task. It combines partitioned bulk extraction, declarative mapping templates, rule-based and statistical quality gates, master data management with deterministic and blocked fuzzy matching, constrained loading with per-record error capture, and change data capture guarded by a data-health circuit breaker. Each stage is mapped to Jitterbit Integration Studio constructs. The framework was implemented as an open reference prototype and evaluated on synthetic two-organization scenarios with known ground truth at three scales, from 32,040 to 383,973 source records, over five random seeds. At a preset threshold, hybrid matching reached pairwise F1 scores of 0.918 for accounts and 0.885 for contacts; calibrating the threshold per object would raise these to 0.938 and 0.963. The quality gates prevented all load failures and all silent loss of unmapped picklist values without wrongly flagging any clean record, whereas removing them produced about 1,000 load failures per run. Over five simulated days of hourly incremental synchronization, replayed events were ignored, an injected upstream fault was contained in the batch in which it occurred, and the target agreed with a full rebuild on more than 99.3 percent of field values. Validation on a production deployment remains future work.

cs.DB · cs.AI · cs.ET

新论文 · v1arXiv:2611.05892

When Is One Query per Template Enough? Adapting Learned Cost Models to New Hardware

Krish Patel

原文摘要

A learned cost model that predicts query runtimes is trained on measurements from one machine. When the database moves to other hardware, its plans stay the same but their runtimes change, each by its own factor. We ask how to adapt such a model from k queries measured on the new machine: which queries to run, how to use their runtimes, and how many are enough. On PostgreSQL we timed the same plans of 1,162 TPC-H and TPC-DS queries in six environments: a laptop and a server, each with all its processors or pinned to one, the server also pinned to two, and the laptop also with a small buffer pool. No single factor describes the slowdown between two environments: most of its variance lies between query templates, so a speed factor stops improving after a few queries. A correction that fits separate lines for serial and parallel plans and adds an offset for every measured template brings the moved model, with one measured query per template, to an error (geometric mean over templates of the median Q-error) of 1.10 on TPC-H and 1.19 on TPC-DS, close to the same correction on a model trained on the new machine; fine-tuning a DACE-style neural cost model on the same queries gains at most 2%. On templates never measured, no correction we tried is on average more than 2% better than the speed factor. We then fixed hypotheses and timed new data. On TPC-H at ten times the scale the errors follow these results, but two of five repeated tests miss. On the Join Order Benchmark, whose queries differ widely within a template, the model errs by 2.36 even when retrained on the new machine, no correction gains much, and only the test on new templates holds; boosted trees and a neural model fit these queries better, and correcting the neural model works as well as fine-tuning it. We report every test as registered.

cs.DB · cs.LG · cs.PF

新论文 · v1arXiv:2611.05882

Optimizing Geo-replicated Deterministic Databases

Yunhao Mao, Shiquan Zhang, Gengrui Zhang, Hans-Arno Jacobsen

原文摘要

In transactional databases, achieving active-active replication among geo-distributed replicas is challenging due to the high coordination cost in ensuring the serializability of concurrent cross-node transactions. Deterministic concurrency control (DCC) has emerged as a promising alternative: it establishes a global order for a batch of transactions before execution and thereby amortizes this coordination cost across the batch. To better understand the performance characteristics of DCC, we conduct a theoretical analysis of traditional two-phase locking and optimistic concurrency control to account for geo-replication and introduce a model for DCC. We compare the three paradigms and identify the conditions under which DCC is advantageous. Guided by the insights from this analysis, we then present Quirinus, a DCC-based geo-replicated database that optimizes transaction execution through speculative execution, transaction reordering, integration of conflict-free replicated data types (CRDTs), and fast-path commit. Even without these optimizations, Quirinus achieves up to 6.2x the TPC-C throughput of Tiga under our configuration.

cs.DB

新论文 · v1arXiv:2611.05871

The Invocation Bottleneck in LLM-Based Lossless Compression: What Can Be Removed and What Must Be Traded

Zhe Yan, Yihao Liu, Antonio Boffa, Huanchen Zhang

原文摘要

Large language models (LLMs) compress text far better than classical compressors, but LLM-based compressors are slow, particularly during decompression. We argue that their decoding time has two fundamentally different sources. Execution redundancy, such as per-token host transfers, CPU-side coding, and token-by-token encoding, is work that the coded output does not require and can be removed without changing a single output bit. Sequential dependency, one model call per decoded token, is inherent to lossless decoding and cannot be removed by better execution, only traded for more calls or for more bits. We remove execution redundancy with a GPU-resident pipeline whose block-parallel encoding is made lossless in half precision by batch-invariant kernels, speeding up arithmetic coding by up to 5.8x in encoding and 3.0x in decoding without changing its compression rate. We then study the two ways of trading sequential dependency, speculative dual-model coding (SDMC) and multi-token single-model coding (MSMC), characterize the compression cost of predicting several tokens from a shared prefix, and identify the workloads in which fewer model calls are worth that cost.

cs.IT · cs.DB

新论文 · v1arXiv:2611.05761

BobFlow: Towards Human-Agent Collaborative Data Science Using Dataflows

Jiadong Bai, Yicong Huang, Chen Li

原文摘要

Recent advances in large language models (LLMs) have established human-agent collaboration as a promising paradigm for data science. Effective collaboration requires a shared computational abstraction that enables agents to reason about processing logic while allowing human analysts to understand, guide, and refine the analysis. Although scripts written in Python or SQL provide precise control, their syntax and execution details can be difficult for analysts to understand and modify. In this paper, we argue that dataflows are an effective abstraction for human-agent collaboration in data science. We introduce BobFlow, a system that combines the dataflow abstraction with the ReAct paradigm. It enables agents to iteratively construct and execute dataflows for data science tasks while allowing analysts to inspect the analysis and provide feedback. Our user study shows that BobFlow helps analysts understand, inspect, and verify agent-generated analyses. A thorough experimental evaluation across proprietary and open-weight LLMs shows accuracy score gains of up to 18.6 points at just over 1/3 of a script-based ReAct agent's inference cost.

cs.DB · cs.AI · cs.HC

版本更新 · v3arXiv:2507.22186

Scalable Source Selection for Machine Learning Tasks

Ambarish Singh, Romila Pradhan

原文摘要

Given a query table, existing data lake discovery techniques seek to identify data sources related to the query table through schema similarity and joinability. The discovered sources, however, are not always useful for downstream analyses; while some improve task generalization, some have the potential to inadvertently introduce harmful noise or distribution shifts. We formalize source selection as the problem of finding a subset of the discovered sources that maximizes downstream task utility. We propose Splice, a target- and cost-aware source selection system that adopts the idea of gene splicing to identify sources which provide additional signals that the union of all sources misses. To speed up the process, we develop two surrogate variants, Splice-DP and Splice-Grad, that estimate the marginal contribution of subsets to task utility by leveraging data profiles in a meta-learning approach and gradient alignment, respectively. On classification tasks over two real-world benchmarks derived from SOTA table union search tools, Splice shows 3-5 percentage points improvement in binary F1 scores compared to the case when all discovered tables are considered. On five additional real-world ML tasks spanning classification and regression, Splice improves task utility over the all-sources baseline by 1.8-13.7 points, while Splice-DP and Splice-Grad match this quality with 3-7x fewer model evaluations (up to 77x at scale).

cs.LG · cs.AI · cs.DB

新论文 · v1arXiv:2611.05690

Turbocharging PiPNN for Proximity and $k$-NN Graph Building

Tobias Rubel, Richard Wen, Guy Blelloch, Laxman Dhulipala, Lars Gottesbüren, Jakub Łącki, Vahab Mirrokni

原文摘要

Graph-based Approximate Nearest Neighbor Search (ANNS) indexes such as HNSW and Vamana deliver state-of-the-art query performance at the cost of large indexing times. The recently introduced PiPNN method drastically accelerated index construction by way of a partitioning-based algorithm leveraging dense matrix multiplication and top-$K$ aggregation kernels. In this paper, we unlock the full potential of PiPNN's hardware-friendly algorithm design. We introduce PiPNN II, which optimizes multiple steps of the PiPNN recipe, giving up to $8\times$ speedups for graph-based ANNS index construction. In addition, we present GPiPNN, a GPU implementation which builds comparable ANNS indices up to $43.7\times$ faster than Jasper and up to $50\times$ faster than CAGRA. We also extend PiPNN II and GPiPNN to compute high-quality $k$-NN graphs directly, which unlocks massive speedups over methods used in many existing systems for clustering, anomaly detection, and more. For 10-NN graph building at $90\%$ recall, we obtain up to a $13\times$ speedup on CPUs and $6.6\times$ on GPUs as compared to the prior best methods. PiPNN II can build a billion-scale 10-NN graph in under 16 minutes. Common to both PiPNN II and GPiPNN is the use of quantization to reduce memory traffic and leverage fast low-precision matrix operations. Top-$K$ emerges as a new bottleneck, which we optimize with specialized code for specific $K$, and by fusing with the matrix kernel to avoid materializing the distance matrix to memory. On CPUs, we demonstrate further performance improvements by leveraging our matrix kernels in the RobustPrune procedure proposed in Vamana and used in PiPNN. On GPUs, we flatten recursive partitioning into segmented batches, use one fused top-$K$ abstraction for partitioning and leaf scoring, and prune with already-witnessed direct edges rather than new candidate-to-candidate scoring passes.

cs.DB · cs.IR

版本更新 · v2arXiv:2607.01546

The General Stability of Ranking

Houming Chen, H. V. Jagadish

原文摘要

Rankings derived from weighted scoring functions are widely used in university rankings and employment candidate evaluations. Since ranking weights are often chosen by organizations or analysts, ranking stability asks whether a reported ranking persists under reasonable weight changes. Prior work defines volume-based stability as the fraction of the weight space that induces the target ranking exactly. This exact-match requirement can be too blunt: once a perturbed weight vector produces a different ranking, exact stability gives it no credit, whether the change replaces the top-ranked item or only swaps two nearly tied lower-ranked items. We propose general stability, a distance-based generalization that aggregates ranking regions according to a user-defined distance from the target ranking. This lets users specify which ranking changes matter in the application, while recovering exact stability as a special case. We extend existing methods and identify distance properties that permit faster general-stability computation with accuracy guarantees. Our sampling algorithm, $GSC_{md}$, can take exponential time in the worst case, but a property we call uniform boundedness gives a sample budget independent of database size and dimension at fixed accuracy and confidence. We develop Prefix-GSC, a sampling optimization that reduces sorting for distances satisfying a further property, tail decay with bounded influence. We also develop Conv-SC to approximate stability through convex volumes for distances satisfying a property we term quasiconvexity. Experiments on seven real-data sources and generated inputs show how distance affects robustness conclusions and when the algorithms save time. On three 500-item university datasets, Prefix-GSC achieves median query speedups of $1.22$--$1.98\times$ over $GSC_{md}$ at 2% relative error and 95% confidence, including method setup.

cs.DB

版本更新 · v2arXiv:2610.36418

When the Application Layer Is Disposable: Putting Integrity Constraints Back in the Database

Hasara Kalumin, Amol Deshpande

原文摘要

Integrity constraints have been central to the relational model since its inception. However, beyond keys, single-row CHECKs, and referential integrity, most constraints are enforced by application layers today. SQL ASSERTIONs, standardized in SQL-92, remain poorly supported by major RDBMSs. This has become increasingly unacceptable due to two developments. First, LLM-driven applications and agents increasingly issue queries and updates directly against databases, bypassing application layers where invariants reside. Such code may be generated on demand and discarded afterwards, making the database the only durable component that can encode and enforce these invariants. Second, natural constraints in higher-level conceptual models, such as E/R and property graphs, become fragmented across relations under mappings chosen for performance. The resulting multi-relation constraints lack direct database support, and only some can be efficiently enforced using existing mechanisms such as triggers. These developments motivate an overhaul of how constraints are defined, mined, managed, and enforced in data management systems. As a first concrete step, we introduce constraint indexes, which represent constraints spanning multiple relations as transactional access paths, and implement a proof of concept as a PostgreSQL extension. Constraint indexes enforce constraints arising from distributed inheritance hierarchies, folded relationships, and folded weak entities, as well as cardinality bounds and functional dependencies, at low overhead relative to no enforcement. Across these workloads, they outperform the better of the correctness-preserving baselines by up to 2.41 times. We also show that LLM-driven writers frequently violate unenforced constraints and behave erratically when provided with a list of constraints to enforce. We close by discussing a few research challenges raised by this agenda.

cs.DB

新论文 · v1arXiv:2611.05568

Non-redundant Independence Atoms

Attila Sali, Jun Yan

原文摘要

Independence atom is a key concept in relational database theory. In the present paper, we prove that the maximum possible size of a non-redundant set of independence atoms on $n$ attributes is asymptotically $(1+o(1))\binom{n}{\floor{n/2}}$. It turns out that this problem is closely related to two extremal set theory problems, the Katona-Tarján question on ``V''-free families of subsets of $[n]$ for the lower bound, and Kleitman's bound on union-free families for the upper bound.

math.CO · cs.DB

新论文 · v1arXiv:2611.06952

Recursively Testing and Vibe Debugging a Google AI Studio Generated Vibe-Coded Database Software Application

Christian Mancas, Cristina Mancas

原文摘要

This research paper proposes a broader definition of vibe coding, extending the original one with non-developers and natural languages other than English, and introduces the notion of vibe debugging, as well as a sound, complete, and optimal algorithmic methodology for testing and vibe debugging vibe-coded software applications. This methodology is illustrated by applying it to a vibe-coded real-world database application of medium complexity generated with Google AI Studio Pro. We prove that even non-developers can obtain a final bug-free and functional satisfactory application version, even in languages other than English. We also prove that, although Google AI Studio Pro has sophisticated built-in engines for both language analysis, software development, and automated testing, it is still making lot of mistakes, so testing and vibe debugging of its generated code is a must.

cs.DB · cs.SE

新论文 · v1arXiv:2611.05173

Correctness Firewalls for Synthesized Database Components

Mohammad Javad Amiri, Carsten Binnig, Andreas Kipf, Tim Kraska, Ryan Marcus, Johannes Wehrstein, Chenyuan Wu, Geoffrey X. Yu 等 9 位作者

原文摘要

For the first time in history, high-performance code is cheap. Today's coding agents can generate bespoke, instance-optimized data systems with excellent performance, and such agents will continue to improve as LLMs advance. But a purely agentic approach, even when validated with strong testing techniques, does not automatically inherit the correctness guarantees our community has spent decades building. We propose a middle ground between hand-crafted and agent-crafted systems: correctness firewalls, lightweight design patterns under which a synthesized component can affect a system's performance but never its results. Behind a correctness firewall, agents are free to synthesize fast components while the overall system remains correct. We present three case studies, each demonstrating a different type of firewall: cardinality estimation (advisory), searching sorted data (guarded), and data compression (certified). In early experiments, synthesized components search sorted data up to 3x faster than human-designed learned indexes and can compress text 1.5x smaller than Zstd. Drawing on these case studies, we outline a research agenda for how data management systems can exploit coding agents without giving up their hard-won correctness guarantees, from protecting synthesized code to measuring the cost of correctness.

cs.DB

版本更新 · v2arXiv:2504.01491

How to Define the Quality of Data? A Feature-Based Literature Survey

Markus Matoni, Arno Kesper, Gabriele Taentzer

原文摘要

Data is generated in almost every digital interaction, and using it effectively requires high quality. But what exactly is data quality? The literature offers many, widely diverging answers. This article presents a systematic literature review that screened more than 8,700 records on data quality and identified 35 providing original, dimension-based definitions. Using feature-oriented domain analysis, a feature model of these heterogeneous definitions is derived and used to classify all identified publications. The classification reveals a heterogeneous landscape: despite an ISO standard and a few highly cited quasi-standards, no consensus exists on which quality dimensions to consider or how to define them. As definitions increasingly emerge for specific domains, contexts of use, and new types of data, the respective communities need to deliberately converge on how data quality is defined in their field.

cs.DB