Ernesto Araya, Gitta Kutyniok, Weilin Li, Johannes Maly, Manjot Singh
In this work, we formalize the concept of intrinsic dimension of objective landscapes introduced in Li et al. (2018), which measures the structural complexity of learning problems by training along randomly generated subspaces. We define the intrinsic dimension of a set $Θ\subset\mathbb{R}^p$ as the minimal dimension $d_{\mathrm{int}}(Θ)$ such that a random plane of dimension $d_{\mathrm{int}}(Θ)$ in $\mathbb{R}^p$ intersects $Θ$ with probability one. We show that it is a generalized notion of co-dimension that depends on the global geometry of $Θ$. We study mathematical properties of the intrinsic dimension and develop techniques to estimate the intrinsic dimension of sets $Θ$ that arise in learning problems. More specifically, we derive upper and/or lower bounds where $Θ$ is the set of optimal parameters in student-teacher settings with shallow and deep ReLU networks, including finite and infinite sample cases, and with and without regularization. We discuss how our theory leads to novel types of generalization bounds, thereby showing that the intrinsic dimension is a measure of learning complexity. Our results connect techniques from seemingly different areas of machine learning research such as classification of symmetries of ReLU networks and stochastic geometry.
The diagonals of a regular pentagon dissect it into eleven pieces and draw a pentagram; the dodecahedron dissected by the twelve face planes of its inner core gives Kramer's seven "elementary cells" of 1982. We define the stellation dissection of a golden regular polytope in any dimension as the dissection of the convex hull of its final regular stellation by the facet hyperplanes of the core, show that the core has phi^(-d) of the inradius of the hull for d = 2, 3, 4, and carry the dissection out for the first time on the 120-cell. The foundation is the arithmetic of the golden ring Z[phi], the ring of integers of Q(sqrt 5): vertices and hyperplanes live in Z[phi]^d, every orientation test is the exact sign of an element of Z[phi], decided with integers, and the pieces are named by sign vectors. We develop this foundation from scratch (conjugation, norm, units, the primes sqrt 5, 2, 3, the exact sign test, the lattice x -> (x, conj x), the vertex modules of the dodecahedron and the 120-cell, the icosian ring) and let it carry the geometry. The definition bifurcates in dimension four: the star cut at depth phi^(-4), a unit, has 2,455,201 cells in 338 orbits of H4; the hereditary cut at depth 1 - phi^(-4) = sqrt 5 / phi^2, a unit times the ramified prime, induces the three-dimensional dissection on every facet and has 8,161 cells in 8 orbits. New here: all vertices of the dodecahedral and of the hereditary dissection lie in the vertex module of the polytope being cut, whereas the star cut needs the denominators 2, 3 and 5; the seven kinds of pieces of the dodecahedron are exactly the I_h-orbits of sign vectors, with stellation layer equal to the number of plus signs. We catalogue the four-dimensional pieces and treat the circumscribed disc, ball and 4-ball. Every number quoted is recomputed exactly by a script whose output is reproduced in an appendix.
Given an arrangement of $\{1,\dots,n^2\}$ in an $n\times n$ grid, we lift each cell to $\mathbb{R}^3$ by centring its grid coordinates $(x,y)$ at the origin and taking its centred entry $z = s_{ij} - (n^2+1)/2$ as a vertical height. The convex hull of these $n^2$ points is the gem of the arrangement, whose lower and upper boundaries are the regular subdivisions of the grid induced by $\pm z$. While random arrangements and generic linear magic squares have gems whose vertex counts grow with $n$, our main theorem shows that the classical Siamese (de la Loubère) magic square of every odd order $n\ge5$ has $f$-vector $(12,26,16)$, isometry group $\{\pm\mathrm{id}\}$, and only the twelve vertices forced by the boundary of the grid. The proof shears the lift onto five horizontal terraces and reduces the hull calculation to a finite table of polynomials in $m=(n-1)/2$; the same method gives $f$-vector $(12,24,14)$ for the base-$n$ digit-exchanged square. Along any family of linear squares with fixed steps the $f$-vector is eventually periodic in $n$, whereas over all linear magic squares of prime order $n$ the maximum vertex count grows as $Θ(\ln n)$. Whenever the heights have vanishing first moments, a short principal-axes argument shows that the point set has a non-trivial isometry if and only if the arrangement is self-complementary up to dihedral symmetry. An exact census of the $880$ magic squares of order four yields $616$ congruence classes and $352$ combinatorial types of gems, and shows that while vertex and facet counts are nearly insensitive to the magic line-sum conditions, the volume and solid centroid shift systematically.
It remains unknown if every prismatoid has a nonoverlapping edge-unfolding, a special case of the long-unsolved "Dürer's problem." Recently nested prismatoids have been settled [Rad24] by mixing (in some sense) the two natural unfoldings, petal-unfolding and band-unfolding. Band-unfolding fails due to a specific counterexample [O'R13b]. The main contribution of this paper is a characterization when a band-unfolding of a nested prismatoid does in fact result in a nonoverlapping unfolding. In particular, we show that the mentioned counterexample is in a sense the only possible counterexample. Although this result does not expand the class of shapes known to have an edge-unfolding, its proof expands our understanding in several ways, developing tools that may help resolve the non-nested case.
We study how boundary geometry affects $L^q$ approximation by shallow ReLU neural networks on convex bodies $K\subset\mathbb R^d$. The target classes are unit balls of weighted variation spaces that can enlarge classical spaces by incorporating domain geometry into constraints on output coefficients. We establish upper and lower bounds for arbitrary convex bodies and determine the optimal polynomial approximation exponents for all planar convex bodies and their products with polytopes, including cubes and circular cylinders. For $C^2$ convex bodies, we determine the optimal exponents in a dimension-dependent range of $q$ extending beyond $2$. When all principal curvatures are positive, the result extends to every finite $q\ge2$. For these classes, the dependence of the optimal exponent on the domain is captured by a single geometric growth exponent associated with small boundary caps. We also identify sharp weight thresholds for retaining the classical approximation exponent and improve previous weights on Euclidean balls while preserving the classical $L^2$ approximation rate. When the target is given as a finite ReLU network, the upper bounds can be realized by a compressed network using only the original neurons, without increasing the weighted coefficient sum. These results show that boundary geometry can change the optimal exponent even when dimension, error norm, and weighting rule are fixed.
The article is devoted to the construction of dimension-raising maps of cubes with the highest possible Holder exponent. In particular, a square-to-cube mapping with a Holder exponent of $2/3$ is constructed, which is a solution to the V.I.Arnold problem.