跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+1205新增论文
+949其中 AI 相关
+153已有论文修订
+59新增 AI 导读

arXiv 最近完成扫描:2026-11-18 04:28 · 全站最近入库:2026-11-18 12:34各信源进度 →

统计口径与数据新鲜度

统计 2026-11-17 13:53 — 2026-11-18 13:53(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-18 04:28最近发现新增版本:2026-11-18 12:34库内论文最新更新时间:2026-11-18 02:59

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

26,054 已收录论文21,703 AI 相关7,828 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 230 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-18 04:27(北京时间)
版本更新 · v2arXiv:2512.24078

高维遗憾最小化

High-dimensional Regret Minimization

Junyu Liao, Ashwin Lall, Mitsunori Ogihara, Raymond Wong

AI 速读 基于摘要

提出 FHDR 框架,利用稀疏线性偏好,通过轻量交互在高维数据上快速发现关键属性,从而降低交互式遗憾最小化的交互轮数与执行时间。

cs.DB · cs.CG · cs.IR

查看原文摘要

Multi-criteria decision making over large databases is fundamental to many real-world applications. Interactive regret minimization has emerged recently as a promising approach that combines the controllable output size of top-$k$ queries with the flexibility of skyline queries, without requiring users to explicitly specify a utility function. Instead, the system iteratively presents a small set of options, learns the user's implicit preferences from their selections, and progressively refines its recommendations. However, modern applications such as housing search and financial product selection often involve hundreds of attributes. Existing interactive methods either fail to scale to such high-dimensional data or require excessive user interaction, frequently exceeding $1000$ rounds. We address this limitation with FHDR (Fast High-Dimensional Reduction), a framework that exploits sparse linear preferences to discover key attributes through lightweight interactions. We also propose AttributeSubset, an efficient single-round algorithm that returns a compact recommendation set without further feedback. In our experiments on real datasets, FHDR completes in under $0.03$s and requires fewer than $30$ interaction rounds under the default setting. Extensive experiments on both synthetic and real-world datasets show that FHDR outperforms the best-known baselines by at least one order of magnitude in execution time and by up to several orders of magnitude in interaction cost. These results establish FHDR as a new state of the art for scalable interactive regret minimization.

版本更新 · v2arXiv:2508.03471

Learned Adaptive Indexing with Updatable Learned Index

Suvam Kumar Das, Suprio Ray

原文摘要

Indexes can significantly improve search performance in databases. However, if the query workload changes frequently or new data updates occur continuously, it may not be worthwhile to build a conventional index upfront for query processing. Building a conventional fine-grained index, like B+-tree on the entire dataset can be very expensive since it may involve processing terabytes to petabytes of data. Moreover, with a traditional indexing, data ingestion tends to slow down in case of heavy updates as the index also needs to be updated. In these cases, adaptive indexing is a viable solution instead of traditional indexing. Adaptive indexing is a technique in which an index gets built on the fly as a byproduct of query processing. In recent years, research has taken on a new direction where machine learning models are employed for the purpose of indexing. These indexes, known as learned indexes, are more efficient compared to traditional indexes in terms of memory footprints and query performance. To the best of our knowledge, no learned indexes exist yet for adaptive indexing. We propose a novel learned approach, called FLAIRS, for adaptive indexing. It is built on the fly as queries are submitted. We also introduce an updatable learned index TULSI that is utilized by FLAIRS for indexing data. FLAIRS can employ a query workload prediction technique that makes future workload projection based on past workload data. We have evaluated FLAIRS against existing adaptive indexes for various query workloads. Our results show that our approach performs better than others in most cases, offering significant improvement in query performance.

cs.DB

新论文 · v1arXiv:2611.12539

Dynamic Graph Indexing for Label-Filtered Approximate Nearest Neighbor Search

Mengxu Jiang, Zhi Yang, Fangyuan Zhang, Yin Yang, Sibo Wang

原文摘要

Approximate nearest neighbor search (ANNS) over high-dimensional vectors is a core primitive in modern retrieval systems. In practice, objects are often also associated with labels (e.g., attributes or keywords), and retrieval must combine vector similarity with explicit label-based filtering, giving rise to label-filtered ANNS. This task is challenging because label filtering and ANNS favor very different index structures, selective filters fragment the admissible search space, and updates complicate index maintenance. To address these challenges, we propose DYLAN, a dynamic composite index for label-filtered ANNS supporting containment, overlap, and their conjunctions and disjunctions. DYLAN organizes label sets by shared prefixes in a frequency-ordered trie and selectively shares HNSW-style graph indices across trie-induced subsets, so that each trie region has a constant-factor covering index while each object participates in only logarithmically many indices. To mitigate fragmentation, DYLAN performs a cross-region multi-graph search that exploits ancestor-level indices as bridges between otherwise isolated trie regions. To support insertions, it adopts a threshold-based maintenance scheme that rebuilds shared indices only when necessary, yielding low amortized update cost. Extensive experiments with real data show that DYLAN consistently improves query throughput at comparable recall, by up to two orders of magnitude on challenging label-rich datasets, while retaining competitive memory, construction, and update costs.

cs.DB

新论文 · v1arXiv:2611.12512

Certifying the Error-Detection Value of Functional Dependencies without Ground Truth

Xiaolong Wan, Xixian Han

原文摘要

Functional dependencies are used to flag erroneous cells, but without error labels the value of these flags is unknown. Dependency strength cannot settle it, because identical data and flags can have positive value in one clean state and negative value in another. We treat this value as partially identified and certify a fixed detector when its value is positive in every clean state compatible with the observation under an error model and budget. The largest such budget is the certification radius. For a detector that flags departures from the unique mode of each class, the worst case reduces to a 0/1 knapsack over class flips, yielding sharp bounds on true positives and the exact radius in polynomial time. Allowing bounded legitimate exceptions in the clean data, we derive an exact threshold for the existence of a positive certificate and compute the radius with the same knapsack. Across five relations and 21,981 candidate dependencies, rules with nearly equal $g_3$ and flag rate can have certification margins differing tenfold. Error injection confirms sound certification and shows that, at equal error counts, changing the organization of errors reverses certification outcomes for every tested rule.

cs.DB

版本更新 · v2arXiv:2610.09903

ERP Event Logs: A Canonical Form for KPI Time-Series Extraction and Characterization

Sherri Hadian, Adrian Rebmann, Atacan Korkmaz, Gregor Berg, Ulf Brackmann

原文摘要

Enterprise resource planning (ERP) systems record operations as transactions spread across hundreds of normalized relational tables. Extracting an operational key performance indicator (KPI) from this schema requires a separate, bespoke join for almost every metric. We show that an event log, a canonical (case, activity, timestamp) representation derived from the same tables, collapses this heterogeneity into one flat structure, from which KPI families reduce to reusable operations after a one-time mapping. Using standardized event logs built from hundreds of SAP S/4HANA customers under a shared activity ontology, we define three families of process-derived KPIs (volumes, durations, and rates) and extract them uniformly across customer systems for two common end-to-end organizational processes, order-to-cash and procure-to-pay. We normalize rates and screen series for minimum coverage to support comparison across customers of very different sizes. We then characterize heterogeneity across industries and tenants using descriptors derived directly from the canonical form, before analyzing the resulting KPI time series. The same process executes heterogeneously across customers and industries, yet this shared representation enables direct comparison of sales- and procurement-side signals, including lead times and order arrivals. Using this shared representation, we characterize which KPI families are forecastable and find a recurring ordering from volumes through durations to rates across both processes and a highly heterogeneous customer base; a pretrained time-series foundation model (Chronos) is competitive with classical baselines such as ARIMA and ETS on rate and duration series, while classical models retain an edge on volumes.

cs.DB

新论文 · v1arXiv:2611.11888

OmniSynth: A Meta-Evaluation Framework for Benchmarking the Quality of Missingness-Inclusive Synthetic Health Data

Annabel Zhu, Océane Doremus, Ariel Guerra-Adames, Marta Avalos-Fernandez, Emmanuel Lagarde, Cédric Gil-Jardiné, Vianney Jouhet

原文摘要

Synthetic data are an emerging solution to the reproducibility crisis in machine learning for health, driven by data privacy restrictions surrounding electronic health records. Yet no consensus exists on how to evaluate synthetic health data quality, existing metrics functionally overlook missingness despite its prevalence in clinical data, and benchmarking frameworks have never been empirically validated. We introduce OmniSynth, a modular Python benchmarking suite that evaluates synthetic tabular data across fidelity, missingness similarity, utility, and privacy - the first framework to explicitly evaluate missingness as a distinct dimension of synthetic data fidelity. OmniSynth further proposes a meta-evaluation procedure that stress-tests evaluation performance under controlled degradation scenarios, providing a systematic characterization of how an evaluation framework behaves across datasets, sample sizes, and variable types. Applying this procedure to OmniSynth in two health datasets (Pima Indians Diabetes; MIMIC-IV-ED), we found that scores responded monotonically to degradation, three of four novel missingness metrics captured induced missing data patterns, and individual metric contributions to the scoring system were empirically justified.

cs.DB

版本更新 · v2arXiv:2502.02709

Enforcing Demographic Coherence: A Harms Aware Framework for Reasoning about Private Data Release

Mark Bun, Marco Carmosino, Palak Jain, Gabriel Kaptchuk, Satchit Sivakumar

原文摘要

The technical literature about data privacy largely consists of two complementary approaches: formal definitions of conditions sufficient for privacy preservation and attacks that demonstrate privacy breaches. Differential privacy is an accepted standard in the former sphere. However, differential privacy's powerful adversarial model and worst-case guarantees may make it too stringent in some situations, especially when achieving it comes at a significant cost to data utility. Meanwhile, privacy attacks aim to expose real and worrying privacy risks associated with existing data release processes but often face criticism for being unrealistic. Moreover, the literature on attacks generally does not identify what properties are necessary to defend against them. We address the gap between these approaches by introducing demographic coherence, a condition inspired by privacy attacks that we argue is necessary for data privacy. This condition captures privacy violations arising from inferences about individuals that are incoherent with respect to the demographic patterns in the data. Our framework focuses on confidence rated predictors, which can in turn be distilled from almost any data-informed process. Thus, we capture privacy threats that exist even when no attack is explicitly being carried out. Our framework not only provides a condition with respect to which data release algorithms can be analysed but suggests natural experimental evaluation methodologies that could be used to build practical intuition and make tangible assessment of risks. Finally, we argue that demographic coherence is weaker than differential privacy: we prove that every differentially private data release is also demographically coherent, and that there are demographically coherent algorithms which are not differentially private.

cs.CR · cs.DB

版本更新 · v2arXiv:2601.03573

Counting Hypertriangles through Hypergraph Orientations

Daniel Paul-Pena, Vaishali Surianarayanan, Deeparnab Chakrabarty, C. Seshadhri

原文摘要

Counting the number of small patterns is a central task in network analysis. While this problem is well studied for graphs, many real-world datasets are naturally modeled as hypergraphs, motivating the need for efficient hypergraph motif counting algorithms. In particular, we study the problem of counting hypertriangles - collections of three pairwise-intersecting hyperedges. These hypergraph patterns have a rich structure with multiple distinct intersection patterns unlike graph triangles. Inspired by classical graph algorithms based on orientations and degeneracy, we develop a new theoretical framework that generalizes these concepts to hypergraphs and yields provable algorithms for hypertriangle counting. We implement these ideas in DITCH (Degeneracy Inspired Triangle Counter for Hypergraphs) and show experimentally that it is faster on all but one small instance, achieving speedups of up to 100x, and is 6-100x more memory-efficient than existing state-of-the-art methods.

cs.DS · cs.DB · cs.SI

新论文 · v1arXiv:2611.11756

Causal Resolution of Undefined Answers in Datalog with Negation

Ratan Bahadur Thapa

原文摘要

Unstratified negation may leave the answer to a Boolean Datalog query undefined under the well-founded semantics. We study database causality in this setting by asking which fact insertions and deletions realize a prescribed transition and which other changes preserve the observed query answer while making a designated fact pivotal. Such a set of changes to other facts is a contingency for the designated fact and the prescribed transition. Our game representation assigns one shared move to each mutable fact and preserves query values and intervention distances. Pairing source and target certificates characterizes all minimum contingencies for the six transitions among true, false, and undefined. When inserting or deleting a move resolves a draw, a directed tree traces the newly winning and losing positions along alternating paths to the source of that move. For the fixed one-rule win--move program, deciding whether a contingency of size at most $k$ exists is $\mathrm{NP}$-complete for each transition, even on bounded-degree graphs. Computing the minimum size is $\mathrm{FP}^{\mathrm{NP}[\log]}$-complete, and counting minimum contingencies is $\#\mathrm{P}$-hard under polynomial-time Turing reductions. When only one of true and false is reachable, the prime implicants of undefinedness satisfied at the observation cover every undefined state, and their supports induce nonempty subtrees of a common tree, we compute a minimum resolving intervention in polynomial time from the game instance. From the explicit tree and these implicants, we construct an equally large family of disjoint supports, certifying optimality.

cs.DB

版本更新 · v2arXiv:2511.11885

Flash-Fusion: Fast, Expressive Queries over Sensor Streams with LLMs

Kausar Patherya, Ashutosh Dhekne, Francisco Romero

原文摘要

Pervasive sensor deployments have generated significant interest in IoT data analysis across fields such as healthcare, transportation and urban planning. At the same time, the rise of Large Language Models (LLMs) offer a new tool for exploring IoT data through their expressive, natural language interface. Unfortunately, natural language querying of IoT sensor streams is impractical today: raw telemetry exhausts LLM context windows in fractions of a second at scale, incurs prohibitive per-token costs, and produces hallucinated responses. Existing systems work around these limits with lossy text summarization or unconstrained code synthesis, producing expensive, unverified computations. Multi-step queries compound this problem: logical errors in intermediate operations can derail the final answer. We present Flash-Fusion, an agentic framework for natural-language querying of tabular IoT data. Flash-Fusion identifies which available data fields and supported derivations can answer the user's query, constructs a plan using typed, domain-specific operators, and validates its structure and schema compatibility before executing. Because constructing plans still requires costly LLM reasoning, Flash-Fusion caches validated operator sequences for reuse across compatible queries. A smaller LLM adapts a cached plan's parameters to each query, cutting latency and cost. We evaluate Flash-Fusion across human activity recognition, ECG monitoring, and vehicular telematics, covering direct lookups, multi-step reasoning, predictive tasks, and out-of-scope requests. Flash-Fusion achieves 92.2% query accuracy versus 76.1% for ReAct, the strongest baseline, while completing queries 7x faster and consuming 5x fewer tokens. Flash-Fusion enables users across disciplines to explore IoT data without manually constructing queries.

cs.DC · cs.AI · cs.DB

版本更新 · v2arXiv:2605.11953

PROTECT-DB: Protecting Data using Replicated State Machines: Efficient Corruption Detection & Recovery

Anant Utgikar, S. Sudarshan

原文摘要

Data is critical for the operation of any organization and needs to be protected, especially against attacks that compromise the state of the database. In this paper, we explore an approach based on Byzantine-fault tolerant replicated state machines, built on top of a deterministic extension of PostgreSQL. Each replica deterministically executes transactions recorded in a shared log/blockchain. Our focus is on creating a practical system that is designed for efficient and quick detection of corruption, as well as quick repair concurrent with execution of transactions. We also present a performance study showing the efficiency and practicality of our approach. We believe our work lays the foundations for the practical use of the BFT replicated state machine approach in the context of databases.

cs.DB · cs.CR

版本更新 · v2arXiv:2610.34248

HNSecW: A Logarithmic-Time MPC Protocol for Approximate Nearest Neighbor Search

Fahong Zhang, Weijie Jin, Lixu Wang, Ning Luo, Xiao Wang, Wei Dong

原文摘要

As a core component of modern RAG systems, approximate nearest neighbor (ANN) search has been extensively studied for decades. In practice, deploying ANN often requires symmetric privacy, where the querier's query and the database owner's data must remain hidden from each other, necessitating a secure multi-party computation (MPC) solution. So far, most MPC-based ANN protocols follow a cluster-based approach: they evaluate distances for the query over a large candidate set and rely on heavyweight primitives such as distributed ORAM to hide access patterns. As a result, each search typically requires $O(\sqrt{N})$ time. We propose HNSecW, a novel MPC protocol for ANN search that achieves the $O(\log N)$ search time for the first time. We first study the single-query setting. Starting from HNSW, the state-of-the-art ANN algorithm, we redesign its control flow and data structures to achieve fully data-independent execution. To further achieve logarithmic search time, we employ offline reshuffling and partial database duplication. Furthermore, we extend our protocol to support multiple sequential queries by introducing a "double visited" caching structure that eliminates redundant cross-query memory accesses, thereby avoiding per-query online shuffling. In addition, we develop a batch-query protocol that exploits higher parallelism to reduce both round complexity and the circuit's critical path. Our experiments show that HNSecW preserves the high accuracy of plaintext HNSW ($\geq 90\%$ recall) and delivers over a $20\times$ efficiency improvement over prior work on million-scale datasets.

cs.CR · cs.DB

新论文 · v1arXiv:2611.11045

Agents Propose, Code Verifies, Humans Decide: Verified Agentic Ingestion of Heterogeneous Scientific Tables into Relational Databases

Bharatha Rankothge, Suzanne Hodgkins, Virginia Rich, Michael Kirby

原文摘要

Public scientific datasets rarely arrive analysis-ready. They arrive as messy folders of wide matrices, long tables, spreadsheets, and per-cohort files whose keys and encodings differ, and every reanalysis must reconnect them first. A coding agent can automate this, but its mistakes can look plausible, and without an audit trail, a mistake that corrupts data can go uncaught. We present an AI-assisted ingestion pipeline built on one rule: LLM agents propose, deterministic code verifies, and humans decide. Narrow agents see compact, citable evidence about the files, never the data, and return typed proposals. A verifier with 39 check types tests each proposal against that evidence in two loops, one for the schema's structure and one for its meaning, and a human reviewer approves the schema and the load. Code compiles PostgreSQL DDL, reshapes the data, quarantines rows it cannot place, records provenance, and reconciles every staged row against the database. On four public biomedical datasets, one held out from development, 9 of 20 first proposals failed a check and every failure was caught: the loops repaired eight, and a one-line reviewer edit fixed the ninth. All 45.1 million values loaded with zero reconciliation gap. Discovering a dataset's structure took 4-7 LLM calls and preparing its data at most one more, costing under $0.50 per dataset. In a separate test, each of nine planted data errors was quarantined with a reason or recorded in an audit log.

cs.DB · cs.AI · q-bio.QM

新论文 · v1arXiv:2611.11031

Hybrid Product Search Without the Copy: Lexical, Semantic and Filtered Retrieval Served Directly from a Data Lake

Raghvendra Yadav, Xiang Fu, Chinmay Soman

原文摘要

E-commerce catalogs increasingly live in lakehouse tables: Apache Iceberg metadata over Parquet files in object storage. Product search, however, still runs on copies of that catalog, in a lexical engine and a vector database, each with its own ingestion pipeline, schema, staleness window and bill. We show that search applications can be hosted directly on the data lake, with the lakehouse table as the single system of record. We extend Apache Pinot, a real-time OLAP engine that already serves Iceberg tables in place, with BM25 relevance scoring over its Lucene text index, corpus-wide term statistics so that scores are comparable across files, and reciprocal rank fusion in plain SQL; a real-time table fed by a stream can sit behind the same table, merged by the broker. On the Amazon ESCI benchmark (1.2M US products, 22,458 test queries), hybrid search served from the lake matches exact retrieval (nDCG@10 .361). The lake table is queryable 149 s after its files are registered, and a lake commit is searchable in 3.8 s, close to indexing directly into a search engine. On a brand-sorted lake of 277M products (0.8 TB), one 16-vCPU node answers brand-scoped vector, BM25 and client-fused hybrid searches in 6.5-9 ms (p50), sustains 1,100 vector and 635 BM25 searches per second at under 180 ms p99, and makes a commit searchable in 4.3 s. Search adds only its indexes: no copy of the catalog, no sync pipeline and no second schema, which cuts storage, rebuild time and the operational work of keeping copies consistent.

cs.LG · cs.DB · cs.IR

版本更新 · v3arXiv:2603.27841

Electrospinning-Data.org, a platform for electrospinning data and analysis

Mehrab Mahdian, Ferenc Ender, Tamas Pardy

原文摘要

Electrospinning produces nanofibers by depositing a polymer solution in an electrostatic field. Fiber morphology is governed by interacting material, solution, process, hardware, and environmental conditions, requiring extensive tuning. Yet observations remain dispersed across the literature in heterogeneous forms that hinder aggregation and reuse. We present Electrospinning-Data.org, the first open, community-oriented research infrastructure dedicated to electrospinning experimental data. Its domain-specific data model links materials, solution properties, process and hardware configurations, environmental conditions, and fiber outcomes, including unstable and unsuccessful experiments. Numeric quantities are harmonized to canonical units and morphology is represented through controlled descriptors. The platform supports guided web, batch, and programmatic contributions, governed by automated structural validation and expert moderation. Its continuously evolving data are complemented by immutable DOI-bearing releases that support reproducible analyses. Public web interfaces, a REST API, and a Python client enable interactive and programmatic access. The integrated Virtual Lab supports exploratory analysis, machine-learning modeling, model interpretation, and evidence-linked parameter exploration. Together, these components establish a failure-aware foundation for reproducible, data-centric electrospinning research.

cs.DB · cond-mat.mtrl-sci

版本更新 · v2arXiv:2512.08274

gHAWK: Local and Global Structure Encoding for Scalable Training of Graph Neural Networks on Knowledge Graphs

Humera Sabir, Fatima Farooq, Ashraf Aboulnaga

原文摘要

Knowledge Graphs (KGs) are a rich source of structured data, and graph neural networks (GNNs) are the dominant tool for learning over them. However, existing message-passing GNNs struggle to scale to large KGs because they rely on the iterative message passing process to learn graph structure. This process is inefficient, especially under mini-batch training, where a node sees only a partial view of its neighborhood. In this paper, we address this problem and present gHAWK, a novel and scalable GNN training framework for large KGs. The key idea is to precompute structural features for each node that capture its local and global structure before GNN training even begins. Specifically, gHAWK introduces a preprocessing step that computes: (a)~Bloom filters to compactly encode local neighborhood structure, and (b)~TransE embeddings to represent each node's global position in the graph. These features are then fused with any domain-specific features (e.g., text embeddings), producing a node feature vector that can be used with any GNN backbone. Equipped with these structural priors, the GNN no longer needs to rediscover graph structure, significantly improving convergence and model accuracy with little memory overhead. Extensive experiments on large datasets from the Open Graph Benchmark demonstrate that gHAWK improves accuracy on both node property prediction and link prediction tasks. Remarkably, gHAWK enables simple GNNs to match or exceed more complex relation-aware ones, suggesting that careful preprocessing can substitute for architectural complexity in KG learning.

cs.LG · cs.DB

新论文 · v1arXiv:2611.10810

Extending DBLog-Based Incremental Snapshots to Handle Orphan Rows in CDC Pipelines

Andreas Andreakis

原文摘要

Applications use heterogeneous datastores where a source feeds sinks such as search indexes or data warehouses via change data capture (CDC). As transaction logs have limited retention, CDC alone cannot bootstrap sinks, and traditional approaches either stall source writes or hold back log events. DBLog-based incremental snapshots, adopted by Debezium and Flink CDC, read tables in chunks interleaved with the log without blocking writes or long log stalls and resume where they stopped. After the initial bootstrap, a sink can drift when losing change events or receiving unintended writes. Even a repeated bootstrap can leave orphan rows that exist in the sink and not at the source. A common remedy is to populate a new empty sink, but repairing an existing sink avoids parallel pipelines, a switch and serving traffic from a drifted sink until the new sink is ready. In DBLog's original design, each chunk only emits the latest state of the rows the table still contains, so it does not handle orphan rows. We extend DBLog to delete them using what a chunk read already knows, namely the keys that the range select did not return, emitting a new kind of event that sinks apply as deletes. We also add a repair request that an external process calls with orphan candidates, which the connector checks with its own read, emitting these events for keys confirmed absent. This enables three modes. In the fully integrated mode, every chunk pass removes the orphans it finds during full or periodic repairs. In the second mode, an external drift process hands suspected orphans to the connector. The third mode combines both and is what we recommend. Together, they turn DBLog into a complete data synchronization engine whether a sink catches up on live changes or missing rows, holds wrong column values, or contains orphan rows. Their safety and convergence are formally verified in Isabelle/HOL and Lean 4.

cs.DB · cs.DC · cs.LO

版本更新 · v4arXiv:2511.14718

Natural Language Interfaces for Databases: What Changes for SQL-Literate Users?

Panos Ipeirotis, Haotian Zheng

原文摘要

Natural Language Interfaces for Databases (NLIDBs) let users query data in everyday language instead of SQL, and modern systems translate most questions accurately. Does an NLIDB remove the work of querying, or move it? We compared SQL-LLM, a GPT-4o-backed NLIDB, with Snowflake, a manual-SQL platform. Twenty SQL-literate professionals and graduate students, ten per interface, each completed 12 BIRD benchmark tasks with hints naming target columns and operators. SQL-LLM cut completion time per query by 31%. It did not improve accuracy: SQL-LLM users answered 46% of queries correctly versus 64% for Snowflake, a gap this sample cannot confirm ($p = 0.08$) that narrows to 14 points without one session where the system timed out. Think-aloud audio from 11 participants (4 SQL-LLM, 7 Snowflake) showed how the work shifted: SQL-LLM users left initial schema navigation to the model, mainly on easier tasks, and focused on checking the generated SQL, while Snowflake users explored the schema and wrote queries by hand. Coded behavior rates did not differ detectably between arms, and positive remarks during a task did not reliably predict a correct answer. The evidence fits a reallocation of querying work, not its removal: users still had to check the generated SQL.

cs.DB · cs.HC

新论文 · v1arXiv:2611.10399

Too Big to Inspect? Ethically Auditing Large Datasets Through Small Samples

Giovana Marchetti, Jhessica Silva, Tiago Torrent, Carlos Caetano, Sandra Avila

原文摘要

Big techs and researchers increasingly claim that their training datasets are too large to inspect for bias and ethical problems. However, in the race to build ever-large models, they have relied on automated web scraping and no human curation. Datasets built this way absorb the stereotypes, exclusionary norms, and historical prejudices of their sources, and the models trained on them can reproduce and amplify those patterns. Because manually auditing billions of samples is humanly impossible, we need practical, effective methods to identify biases in these datasets. This paper studies the feasibility of ethically auditing large datasets through small samples and proposes a practical method for identifying structural biases by applying AI Ethics Tools (AIETs) to random samples. We applied Datasheets for Datasets, Data Statements, Data Nutrition Labels, and Data Cards to a 1% sample of Framed Multi30K, and Datasheets for Datasets to two 0.00006% samples of relaion2B-en-research-safe. Our results show that evaluating small samples with AIETs can reveal ethical problems, structural biases, and technical inconsistencies. Although industry practice assumes that identifying problems requires analyzing the entire dataset, we show that a random subset supports a viable qualitative analysis without overwhelming computational or human resources. Demonstrating that a dataset is biased does not require examining every instance: if a bias appears in a random sample, it is a flaw of the creation process. The AIETs' questions acted as provocateurs, evoking different perspectives and analysis methods for the same dataset, but the analysis and its results depended fundamentally on the human factor. We hope these findings contribute to urgent public discussions about ethics, transparency, accountability, and AI regulation by offering a practical pathway for responsibly auditing massive datasets.

cs.AI · cs.DB

新论文 · v1arXiv:2611.10242

LoStBench: Benchmarking Longitudinal Studies Data Integration

Pratik Pokharel, Oliver Kennedy, Abhi Ramtel, Poonam Kumari, Jeff Good, Katarzyna Kordas, Marianthi Markatou, Andrew Talal

原文摘要

A longitudinal study observes the same subjects or variables, repeatedly over an extended period of time, allowing researchers to examine changes, trends, and relationships as they develop. Such studies typically evolve over their life cycles, as improved understanding of the study's domain, or external factors trigger the need for revisions. The resulting heterogeneity among collected data must be resolved before the survey's results may be analyzed. Although this problem is a form of schema matching, it is a form of schema matching that differs significantly from prior workloads explored by the database community. For instance, rather than the short attribute names and data instances that many schema matching tools rely on, a stronger signal in longitudinal studies arises from longer prose questions that these tools are less equipped to handle. This paper documents the joint efforts of a team of computer scientists and domain experts to adapt existing schema matching algorithms to our new use case of longitudinal data integration. We present a case study in the context of a representative workload: The American National Election Studies (ANES) dataset, a nearly eight-decade survey with publicly available data and a canonical domain-expert curated schema matching. Using ANES, we develop LoStBench (Longitudinal Studies Benchmark for data integration), a medium-scale, subject-matter-expert-annotated benchmark. Using LoStBench, we conduct a comprehensive empirical study evaluating state-of-the-art schema matching algorithms, and expose limitations of these algorithms when applied in the context of longitudinal studies. We propose improvements that address some of these limitations, and identify key challenges to motivate future data integration work to address the others.

cs.DB

版本更新 · v3arXiv:2512.22838

超越重叠:面向外存向量搜索的分层图抽象

Beyond Overlap: Hierarchical Graph Abstraction for Out-of-Core Vector Search

Lizheng Chen, Pinhuan Wang, Shaonan Ma, Jie Zhang, Qing Wang, Zhengyi Yang, Heng Zhang, Mingxing Zhang 等 12 位作者

AI 速读 基于摘要

提出分层外存 ANNS 引擎 GANNS,用内存驻留的图抽象在 SSD 访问前实现逻辑重叠,兼顾精度与 I/O 效率。

cs.DB · cs.IR

查看原文摘要

At billion scale, approximate nearest neighbor search (ANNS) often becomes an out-of-core problem: the full vector collection and index structures exceed memory capacity, making query performance dominated by SSD accesses and DRAM-SSD data movement. Existing systems struggle to strike a balance between accuracy and efficiency: physical-overlap methods replicate vectors or index entries across partitions, enlarging the SSD-resident index and incurring extra I/O; quantization-based methods reduce memory usage, but their approximate distances are less reliable and often require costly raw-vector reranking from SSD to preserve recall. We present GANNS (Graph-Abstraction ANNS), a hierarchical out-of-core ANNS engine that improves accuracy and I/O efficiency simultaneously. GANNS stores each cluster as a disjoint SSD partition with scale-aware indexes, while a memory-resident graph abstraction provides logical overlap before SSD access. During serving, GANNS uses query hotness and cluster priorities from the graph abstraction to prune low-value clusters and improve access locality. Across five datasets under strict memory constraints, GANNS delivers representative QPS improvements of 2.0--4.7x over state-of-the-art baselines, while preserving accuracy.

版本更新 · v2arXiv:2607.00768

RACORN: Robust Predicate-Agnostic Filtered Vector Search under Extreme Selectivity

Yoonseok Kim, Gyusik Choe

原文摘要

Filtered vector search returns the nearest neighbors among the vectors that pass a predicate. The predicate is often selective: most tag queries of the Big ANN filtered track select less than 1% of YFCC 10M. Predicate-agnostic methods search one HNSW graph built without metadata and so take any predicate, but below a few percent selectivity they break down. ACORN-1 draws candidates only from passing neighbors and its recall collapses below 1%. HNSW keeps searching but spends about ef/s expansion cycles at selectivity s. We present RACORN-1, an extension of ACORN-1 that keeps its graph and filter-first traversal and adds Adaptive Search Fallback. When the two-hop expansion of a node falls short of a target, failing nodes enter the candidate queue as bridges, under a per-expansion budget, but never enter the result queue. RACORN-1 serves the selectivities ACORN-1 cannot: from 1% down ACORN-1 no longer reaches recall 0.9 at any ef, while RACORN-1 reaches 0.95 from 3% down on every dataset we sweep. It also reaches 0.9 on every natural filter that YFCC and LAION ship and on every conjunction and caption predicate we run, where ACORN-1 misses more than half. With a deadline equal to the time HNSW needs to fill its result queue, RACORN-1 returns the higher recall on all eight datasets at four selectivities. On eight indexes of 1M to 100M vectors, at 1% selectivity and recall 0.9 it is 1.3-8.4x faster than HNSW, and it is 2.2-10.5x faster under adverse correlation.

cs.DB · cs.IR

版本更新 · v2arXiv:2501.08686

Knowledge Graph-based Retrieval-Augmented Generation for Schema Matching

Chuangtao Ma, Sriom Chakrabarti, Arijit Khan, Bálint Molnár

原文摘要

Traditional similarity-based schema matching methods fail to resolve semantic ambiguities and conflicts in complex, domain-specific mappings due to a lack of commonsense and domain-specific knowledge. In parallel, large language models (LLMs) without sufficient background knowledge tend to hallucinate, further complicating schema matching. To overcome these limitations, we introduce KG-RAG4SM, a Knowledge Graph-based Retrieval-Augmented Generation for Schema Matching. This method integrates novel retrieval mechanisms, including LLM-based, vector-based, graph traversal-based, and hybrid techniques with advanced reranking schemes to extract the most relevant subgraphs from large external KGs. Our approach achieves superior results without requiring re-training, as evidenced by improvements of 39.79% in precision and 30.56% in F1 score over LLM-based state-of-the-art (SOTA) methods (e.g., Jellyfish-8B) on the MIMIC dataset, and enhancements of 69.20% in precision and 21.99% in F1 score over pre-trained language model (PLM)-based SOTA methods (e.g., SMAT) on the Synthea dataset. Additionally, KG-RAG4SM scales effectively across KGs from commonsense KGs to domain-specific ones, while its instruction-tuning knowledge exclusion component ensures truthful performance in end-to-end schema matching tasks despite the suboptimal relevance of KG context. Our empirical analysis and case studies from real-world scenarios show that the hallucination problem in LLMs for schema matching is well mitigated by our solution.

cs.DB · cs.CL · cs.IR

新论文 · v1arXiv:2611.09372

Enabling Privacy-Preserving Natural Language Access to Industrial Smart Meter Data

Carlos Martín-Abellán, Mark Kröll

原文摘要

Energy management officers and grid operators increasingly rely on smart meter data for anomaly detection, performance monitoring, and strategic decision-making. However, extracting actionable insights from large-scale, heterogeneous telemetry databases requires advanced SQL and data engineering expertise, creating a critical bottleneck for non-technical personnel. Furthermore, the strict data privacy regulations and security constraints inherent to critical energy infrastructure often prohibit the routing of sensitive consumer data through cloud-based Large Language Models (LLMs). To address these challenges, we present OpenMeter, a schema-agnostic Multi-Agent System (MAS) designed to translate natural language queries into executable code and human-readable insights. This architecture lowers the technical barrier for day-to-day utility operations by empowering non-expert personnel to interact with complex databases through intuitive natural language dialogue, transforming raw telemetry into a highly valuable tool for routine grid workflows and strategic decisions. Multi-scenario ablation studies over this environment demonstrate that full multi-agent orchestration shows its largest observed accuracy gains on resource-constrained local hardware (+6.1 pp EX-R over the strongest single-agent configuration); at frontier scale, individual components retain value but should be enabled selectively, indicating that the cost of full orchestration is justified specifically where privacy demands local execution.

cs.AI · cs.CL · cs.DB · cs.MA

新论文 · v1arXiv:2611.09346

Logical Scalability in Big Data Architectures: Impact of Data Modeling on the Database Layer

Rosario Napoli, Mark Adrian Gambito, Antonio Celesti, Maria Fazio

原文摘要

In modern Big Data architectures, scalability is often addressed through infrastructural expansion, involving vertical or horizontal scaling to manage growing data volumes. However, this hardware-centric perspective overlooks the logical scalability of the storage layer. A central challenge lies in how data modeling choices affect the storage layer of big data architectures. Specifically, when the data model is misaligned with the underlying query patterns, performance bottlenecks emerge and persist regardless of the available computational resources. This paper investigates the concept of logical scalability by conducting a comparative performance analysis of Document, Graph and Multi-Model database paradigms under varying workloads, queries and dataset scales. Our results demonstrate that performance degradation caused by model misalignment cannot be mitigated by additional computational resources alone. Increasing the number of machines while retaining a structurally inadequate data model yields performance equivalent to or worse than a single-machine configuration, as distributed coordination overhead compounds the inefficiencies of the underlying logical model. We demonstrate that efficiency is maximized when the logical structure of the database aligns with the inherent nature of the data and the intended analysis scope, a condition that proves to be more decisive than any infrastructural scaling strategy.

cs.DB

新论文 · v1arXiv:2611.09172

PaB-tree: Parallel B+-tree with Batch Updates

Ziyang Men, Yan Gu, Yihan Sun, Helen Xu

原文摘要

The B+-tree is a ubiquitous search index due to its provable I/O (cache) efficiency. Yet unlike batch-parallel binary search trees, which are both fast and widely available, no batch-parallel B+-tree offers strong empirical performance together with provable, near-optimal I/O efficiency. Prior B+-tree-based designs are either parallel but purely theoretical, with no high-performance implementation, or concurrent indexes such as the Bw-tree and Masstree, which run an order of magnitude slower on batch updates. We present PaB-tree, the first batch-parallel B+-tree with guarantees on work, span, and cache complexity alongside strong practical performance. Its core challenge -- repairing node overflows and underflows without eroding the tree's I/O advantage -- is resolved by one simple invariant: a repair never touches a sibling and is always pushed to the parent. A Promote primitive reduces batch construction and insertion to repeated node promotion, and batch deletion to reinsertion. Batch updates take O(B log_B n) work and O(log_B n) I/O per element, matching the sequential B+-tree bounds, with polylogarithmic span. Against highly-optimized parallel and concurrent baselines, PaB-tree delivers the best overall performance trade-off: up to 1.4x faster on construction and up to 3.6x faster on batch insertion than the strongest baseline, while preserving B+-tree query efficiency.

cs.DS · cs.DB · cs.DC

新论文 · v1arXiv:2611.09065

MongoBench: Do LLMs Know What a MongoDB Query Will Do?

Qibai Chen, Haoting Qiu, Yezhou Cheng, Mengxiao Qian

原文摘要

LLM agents increasingly operate databases on their own: they generate queries and run them against live data. Two checks are commonly used to judge such an agent - whether it produces a valid query, and whether it reaches the right final answer - but neither tells us whether the agent knows what its query will actually do before it runs. We argue that this consequence prediction ability must be evaluated in its own right, and we build MongoBench to measure it: given a collection of documents and a MongoDB find() filter, a model must return the exact set of matching _ids, with ground truth from a real mongod server. The 300 items each compose three or four documented matching pitfalls and are proposed symmetrically by three vendors' models, so results stratify by item source. Evaluating four models - frontier systems from three vendors and one open-weight reasoner - accuracy ranges from 70.1% to 92.1%, a 22-point spread. Each proposer mispredicts 10-31% of the items it wrote itself, so producing a query does not imply predicting its effect. A failure-mode analysis shows errors are better predicted by which semantic pitfall a query exercises - range comparisons and null - than by surface measures such as document count or the number of composed rules. Prepending a verified rule sheet helps unevenly; executing the original filter against the engine yields reliable match sets, but delegation must be checked: on 28% of items (averaged over three runs) one model's executed filters never passed a canonical check against the intended query, and it answered about 22 of them wrong, while the other three passed on every item. We release the benchmark, the mongod oracle, and the evaluation code with all prompts at https://github.com/research-repo-bai/MongoBench.

cs.SE · cs.DB

版本更新 · v2arXiv:2611.01454

PROBE: Learned Proxy-Based Execution for Highly Selective Multimodal Semantic Queries

Haikai Zhao, Haisheng Fu, Zhengjie Miao, Zhenman Fang

原文摘要

Large language models (LLMs) enable semantic queries over unstructured data, but invoking an LLM on every row is prohibitively expensive. Proxy-based execution can reduce this ple with the LLM and training a lightweight proxyto label the remaining rows. Nevertheless, in practical scenarios where semantic queries are highly selective, it remainschallenging to identify representative samples foLM budget and to achieve accurate predictions overthe remaining data using the proxy. We present PROBE, a learned proxy-based execution framework for highly selective multimodal semantic queries. PROBE uses selectivity-adaptive sampling to discover rare polightweight proxy on the acquired labels. It thenrefines proxy predictions through residual correction and homogeneity smoothing over a similarity graph. To address sampling bias, PROBE jointly calibrates refined scores usiarate calibration sample, then selectively routescandidates to the LLM by optimizing the tradeoff between estimated F1 and evaluation cost. On queries from SemBench and LRO-Bench with positive rates as low as 2.17x10^-5 budget yields a geometric-mean F1 of 0.762,compared with 0.797 for exhaustive LLM evaluation, at a geometric-mean speedup of 4.87x and a maximum speedup of 9.3x.

cs.DB

新论文 · v1arXiv:2611.09052

StreamTab: Training-Free Temporal Link Prediction with Tabular Foundation Models

Mingqi Yang, Renqi Wu

原文摘要

Temporal link prediction uses past interactions to predict whether two nodes will interact in the future. Temporal graph neural networks achieve strong results on this task, but they are trained separately for each graph. How a model should use interaction history also depends on the candidates it is compared with. A past interaction can distinguish a recurring link from a random pair that has never interacted, but when the negative candidates have also interacted before, other signals are needed. We propose StreamTab, a training-free framework built on a frozen tabular foundation model pretrained only on synthetic tables. StreamTab describes each candidate pair by statistics of its past interactions and shows the model recent observed interactions and sampled negative pairs as labeled examples, from which the model scores new candidates. Its central design, regime conditioning, samples the negative examples according to how the candidates are sampled, so the examples show the distinction the model must make. As new interactions arrive, StreamTab updates the statistics and refreshes the examples while the model parameters stay fixed. On thirteen datasets against 21 temporal graph baselines, StreamTab achieves the highest average precision on twelve datasets under historical sampling and eleven under inductive sampling, and ranks first or stays within one point of the best on seven datasets under random sampling. With the same features and model, regime conditioning raises average precision over examples with random negatives by a median of 12.6 points under historical sampling.

cs.LG · cs.DB · cs.SI

新论文 · v1arXiv:2611.09040

MV-IVF: Multi-Vector Retrieval via Multi-Vector Clustering

Kishen N Gowda, Laxman Dhulipala, Lars Gottesbüren, Rajesh Jayaram

原文摘要

Multi-vector retrieval represents each query and document as a set of embeddings and scores relevance via fine-grained set-to-set interactions such as MaxSim (Chamfer similarity). This expressiveness improves retrieval quality over single-vector representations, but makes search substantially more expensive. Existing systems address this challenge primarily through proxy representations or filter-and-rerank pipelines that reduce candidate generation to single-vector search. We study the complementary direction of building a native IVF-style index directly over multi-vector data. We introduce MV-IVF, an inverted-file index for multi-vector retrieval built on MV-Lloyd, a new Lloyd-style algorithm for clustering point clouds under a Chamfer objective. Our key observation is that once point clouds are assigned to clusters, the center-update step reduces to standard single-vector clustering over the flattened embeddings in each cluster. This enables efficient construction of a hierarchical IVF tree with center point clouds at internal nodes. At query time, MV-IVF performs best-first beam search over the tree to identify promising leaf clusters, then scores the documents in those leaves. We further accelerate search using low-bit quantization, query compression, and cache-efficient batched search. We implement MV-IVF in MVSIC, an optimized C++ library and unified evaluation framework for multi-vector retrieval that includes several major approaches. Across BEIR and LoTTE text retrieval benchmarks, MV-IVF achieves strong latency--recall tradeoffs, the best batched throughput frontiers, and favorable build-time and index-size profiles. At $90\%$ Recall-$10$@$10$, MV-IVF achieves up to $8.03\times$ lower single-query latency and up to $173.5\times$ higher QPS than the baselines, demonstrating that native IVF-style indexing is practical and effective for scalable multi-vector retrieval.

cs.IR · cs.DB