跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.10398 · v2

Measuring Tabular Data Formats Cold: A Replicated Cold-Cache Benchmark of Serialization, Scan, and Index Performance on Real Datasets

Asra FathimaKristoffer Roberts
cs.DBcs.PFcs.SE
首次提交 2026-11-07 10:10
版本更新 2026-11-11 00:09
雷达观测 2026-11-11 13:07
arXiv ↗PDF ↗

摘要

Choosing a file format for tabular data is a routine engineering decision, yet published comparisons often time reads served from the operating-system page cache, use one session, or rely on a single synthetic dataset. This article benchmarks fifteen format and access-path configurations (CSV, JSON Lines, Pickle, Feather, Parquet, ORC, DuckDB, and SQLite, with several access paths) on two real public datasets, New York City taxi trips and United States airline on-time records, at 50,000 to 3 million rows. Every measurement uses an explicitly cold cache, randomized execution order, and three independent sessions (with restarts on the laptop), and is analyzed with bootstrap intervals and Holm-adjusted permutation tests. Parquet answered a filtered aggregation 2.6 to 11 times faster than CSV, while JSON Lines was 10 to 27 times slower. A secondary index made SQLite queries 2.6 to 4.8 times slower on randomly ordered data and helped only on data sorted by the filtered column, at low selectivity. A row filter inside the Parquet reader was significantly faster than a projected scan on flight data from one million rows and on both datasets at three million, and row-group skipping needs several row groups, which the default writer produced only at 3 million rows. A 4 GB Linux host ran out of memory on configurations that a 24 GB laptop completed and reversed some orderings, such as Pickle versus Parquet. Code, derived data, and raw timings are released.

作者备注：Submitted to IEEE Open Journal of the Computer Society

查看 arXiv 原文 ↗阅读 PDF ↗

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存，登录后共享已有结果。

版本差异

仅对比实际观测的元数据，不代表 PDF 正文或实验变化。版本可能不连续；带删除线为移除，下划线为新增。

标题变化

Comparing Write, Read, and Query Performance of General-Purpose andMeasuring ColumnarTabular Data Serialization Formats at Increasing ScaleCold: AnA EmpiricalReplicated Cold-Cache Benchmark Studyof withSerialization, StatisticalScan, Validationand onIndex SyntheticPerformance andon Real-World DataDatasets

作者变化

Asra Fathima,Kristoffer Roberts

摘要变化

This study presents an empirical benchmarkChoosing comparinga ninefile format andfor access-methodtabular combinationsdata acrossis sixa dataroutine serializationengineering technologies:decision, CSVyet (includingpublished acomparisons column-projectedoften variant),time gzipreads CSV,served JSONfrom Lines,the Pythonoperating-system Pickle,page SQLitecache, (unindexeduse andone indexed)session, andor Apacherely Parqueton (naivea andsingle pyarrowsynthetic pushdown)dataset. AThis syntheticarticle datasetbenchmarks wasfifteen generatedformat atand fouraccess-path scalesconfigurations (50CSV,000 toJSON 3,000Lines,000 rows)Pickle, with write timeFeather, storage sizeParquet, readORC, timeDuckDB, and query timeSQLite, measuredwith acrossseveral twentyaccess trialspaths) on two independentreal hostspublic datasets, aNew nativeYork 24City GBtaxi Appletrips Siliconand machineUnited andStates aairline memoryon-constrainedtime records, single-coreat 450,000 GBto Linux3 virtualmillion machinerows. Mean,Every standardmeasurement deviation,uses andan 95explicitly percentcold confidencecache, intervalsrandomized areexecution reportedorder, forand eachthree host.independent Thesessions most(with robustrestarts finding,on stablethe acrosslaptop), everyand scaleis andanalyzed bothwith hosts,bootstrap isintervals thatand flatHolm-adjusted formatspermutation requiringtests. fullParquet deserializationanswered beforea filteringfiltered areaggregation one2.6 to three orders11 oftimes magnitudefaster morethan expensiveCSV, forwhile selectiveJSON queriesLines thanwas any10 approachto that27 avoidstimes itslower. Welch t-tests,A corroboratedsecondary byindex Mann-Whitneymade USQLite tests,queries show2.6 SQLiteto (indexed)4.8 istimes significantlyslower fasteston forrandomly selectiveordered accessdata atand everyhelped scaleonly on bothdata hosts,sorted withby the pushdown path of Parquet significantlyfiltered cheapercolumn, thanat naivelow materializationselectivity. With theA increasedrow trialfilter count,inside the write-timeParquet variancereader ofwas Picklesignificantly narrowedfaster enoughthan toa showprojected itscan ison significantlyflight fasterdata thanfrom Parquetone atmillion writerows timeand on both hosts,datasets notat thethree statisticalmillion, tieand anrow-group earlierskipping five-trialneeds pilotseveral suggested.row Thesegroups, findingswhich werethe confirmeddefault onwriter aproduced real,only publicat 1.47-3 million-row e-commercerows. fraudA dataset,4 whichGB reproducedLinux thehost sameran order-out of-magnitude gapmemory betweenon flatconfigurations andthat non-flata formats24 andGB thelaptop samecompleted SQLiteand (indexed)reversed advantage.some Theseorderings, resultssuch offeras statisticallyPickle groundedversus guidanceParquet. forCode, formatderived selectiondata, inand dataraw engineeringtimings pipelinesare released.

作者备注变化

12 pages, 5 figures, 10 tables. Submitted to IEEE AccessOpen (ManuscriptJournal IDof Access-2026-37666)the Computer Society

观测到的版本

相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-11 00:09 · 最新观测版本

Measuring Tabular Data Formats Cold: A Replicated Cold-Cache Benchmark of Serialization, Scan, and Index Performance on Real Datasets

Choosing a file format for tabular data is a routine engineering decision, yet published comparisons often time reads served from the operating-system page cache, use one session, or rely on a single synthetic dataset. This article benchmarks fifteen format and access-path configurations (CSV, JSON Lines, Pickle, Feather, Parquet, ORC, DuckDB, and SQLite, with several access paths) on two real public datasets, New York City taxi trips and United States airline on-time records, at 50,000 to 3 million rows. Every measurement uses an explicitly cold cache, randomized execution order, and three independent sessions (with restarts on the laptop), and is analyzed with bootstrap intervals and Holm-adjusted permutation tests. Parquet answered a filtered aggregation 2.6 to 11 times faster than CSV, while JSON Lines was 10 to 27 times slower. A secondary index made SQLite queries 2.6 to 4.8 times slower on randomly ordered data and helped only on data sorted by the filtered column, at low selectivity. A row filter inside the Parquet reader was significantly faster than a projected scan on flight data from one million rows and on both datasets at three million, and row-group skipping needs several row groups, which the default writer produced only at 3 million rows. A 4 GB Linux host ran out of memory on configurations that a 24 GB laptop completed and reversed some orderings, such as Pickle versus Parquet. Code, derived data, and raw timings are released.

查看此版本解读 →查看此版本原文 ↗
v1 · 2026-11-07 10:10

Comparing Write, Read, and Query Performance of General-Purpose and Columnar Data Serialization Formats at Increasing Scale: An Empirical Benchmark Study with Statistical Validation on Synthetic and Real-World Data

This study presents an empirical benchmark comparing nine format and access-method combinations across six data serialization technologies: CSV (including a column-projected variant), gzip CSV, JSON Lines, Python Pickle, SQLite (unindexed and indexed), and Apache Parquet (naive and pyarrow pushdown). A synthetic dataset was generated at four scales (50,000 to 3,000,000 rows), with write time, storage size, read time, and query time measured across twenty trials on two independent hosts, a native 24 GB Apple Silicon machine and a memory-constrained, single-core 4 GB Linux virtual machine. Mean, standard deviation, and 95 percent confidence intervals are reported for each host. The most robust finding, stable across every scale and both hosts, is that flat formats requiring full deserialization before filtering are one to three orders of magnitude more expensive for selective queries than any approach that avoids it. Welch t-tests, corroborated by Mann-Whitney U tests, show SQLite (indexed) is significantly fastest for selective access at every scale on both hosts, with the pushdown path of Parquet significantly cheaper than naive materialization. With the increased trial count, the write-time variance of Pickle narrowed enough to show it is significantly faster than Parquet at write time on both hosts, not the statistical tie an earlier five-trial pilot suggested. These findings were confirmed on a real, public 1.47-million-row e-commerce fraud dataset, which reproduced the same order-of-magnitude gap between flat and non-flat formats and the same SQLite (indexed) advantage. These results offer statistically grounded guidance for format selection in data engineering pipelines.

查看此版本解读 →查看此版本原文 ↗