Anders Aamand, Justin Y. Chen, Mina Dalirrooyfard, Slobodan Mitrović, Sandeep Silwal, Yinzhan Xu
原文摘要
We study the problem of privately releasing the min cut values of all pairs of vertices in a given $n$-node weighted graph, under the edge-neighboring model. Straightforward application of advanced composition yields an approximate DP algorithm that outputs all min cut values up to $\tilde{O}(n)$ additive error. Prior works [Dalirrooyfard, Mitrovi{ć} and Nevmyvaka, 2023; Aamand et al., 2026] have established that $\tildeΘ(n)$ is the correct bound if the algorithm also needs to output cuts that realize the approximate min cut values. On the other hand, simply outputting the values could be an easier problem, but prior algorithms have been stuck on the $\tilde{O}(n)$ bound. In this work, we give an approximate DP algorithm outputting all-pair min cut values with $\tilde{O}(\sqrt{n})$ additive error, beating the linear error barrier. Our algorithm applies the sparse vector technique [Dwork and Roth, 2014] to maintain a hierarchical decomposition of the vertices based on the min cut structure. We also prove an $Ω(\log^{3/2} n)$ lower bound for this problem.
Justin Y. Chen, Ying Feng, Piotr Indyk, Michael Kapralov, Ekaterina Kochetkova, Boris Prokhorov
原文摘要
A growing body of work on KV-cache compression aims to reduce the memory requirements of autoregressive generation. These methods compress the stored context while approximately preserving attention computation, enabling generation with reduced memory consumption. Almost tight theoretical results on the worst case space complexity of attention have recently been obtained, but they are achieved by instances where queries can be perfectly aligned with keys. This directly contradicts the observed anisotropy of the key/query geometry in real transformers. In particular, it has been shown empirically that key/query cosine similarities are small even for most correlated pairs, i.e., near-perfect alignment of queries with keys does not happen in practice. In this paper, we study the space complexity of attention approximation under the assumption that key/query cosine similarities belong to a small interval around zero, i.e., a query cannot align too much with any key. We give a nearly space optimal algorithm and a corresponding space lower bound. Surprisingly, the distribution of key/query cosine similarities in our lower bound exhibits the anisotropy property empirically established and studied in the literature. Additionally, we show that the simple uniform sampling algorithm is nearly optimal for a typical range of parameters, supporting its surprising effectiveness on real datasets.