Sliding-Window Quotas in Apache Hadoop YARN: Exact Admission and Flexible Placement
Ruize Zhao
原文摘要
A reservation in Apache Hadoop YARN can pass a quota of 600 MiB-hours even when a 24-hour window contains 720 MiB-hours of usage. YARN subtracts sampled prefixes to estimate the window total. This can also reject requests that fit, and adding demand can introduce a sample that reverses a rejection. We bound the peak error in both directions and implement an exact check that visits rate changes at either edge of the window. When the two policies see the same history in replays from two public job traces, they disagree on up to $13.3\%$ of requests. Among requests both policies admit, the 95th-percentile latency of inserting a reservation into the plan is $39$--$47\%$ of the sampled baseline. A separate linear scan finds all feasible integer starts for a single constant-demand stage. In a ResourceManager test, the default agent rejects a flexible request. The ResourceManager accepts the stage when submitted at a start found by the scan. In one replay, committing such retries reduces total admissions.
cs.DS