Igor Shalygin, Grigorii Davydenko, Andrey Veprikov, Aleksey Antonov, Nikolay Bushkov, Stanislav Moiseev, Aleksandr Beznosikov
The success of large language models in real-world applications relies on parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA), which inserts compact, trainable adapters into selected layers. In this paper, we propose SimplexLoRA, a LoRA variant that adaptively adjusts adapter ranks during fine-tuning based on their learned importance. Unlike standard LoRA, which employs a fixed rank across all layers, SimplexLoRA optimizes per-adapter importance weights on the $n$-scaled simplex during a short warm-up and then freezes them, dynamically allocating capacity where it is most beneficial without increasing the overall parameter budget or inference cost. We evaluate SimplexLoRA across 12 language understanding, summarization, and reasoning tasks using DeBERTaV3, BART-large, Llama 3.1 8B, and Qwen3-8B. Our results show that SimplexLoRA consistently matches or outperforms standard LoRA and recent adaptive baselines, with the largest gains at low ranks. Our code is available at https://github.com/brain-lab-research/SimplexLoRA.
We study the \emph{Polynomial Avoidance Problem}: given a finite collection $F \subseteq Z[x_1,\dots,x_n]$ of nonzero polynomials, find a vector $x \in Z^n$ of minimum $\ell_p$-norm such that $f(x)\neq0$ for every $f \in F$. Each polynomial is given sparsely as a list of its nonzero terms. The affine special case is the \emph{Hyperplane Avoidance Problem}. This special case arises in Barvinok-type algorithms for counting integer points in polyhedra, when certain rational generating functions must be evaluated at a removable singularity. Our main results are as follows: 1) Let $D:=\sum_{f\in F} \text{deg} f$. Every instance admits a feasible vector $x$ satisfying \[ \|x\|_1 \leq \min\left\{D,\frac{D+n}{2}\right\}. \] This bound is tight. Such a vector can be found by a deterministic algorithm whose bit complexity is polynomial in $D$ and the sparse encoding length of the input. Our proof uses a new algorithmic variant of the \emph{Combinatorial Nullstellensatz}; 2) In the affine case, with $m$ input polynomials, such a vector can be constructed by a fast deterministic polynomial-time algorithm using $O(nm)$ arithmetic operations. For the homogeneous case, Barvinok's original approach provides only the weaker guarantee $\|x\|_1 = O((nm)^{n-1})$; 3) The problem is NP-hard for every $p\in R_{\geq 1}\cup\{\infty\}$, even for homogeneous linear forms; 4) As an application, we improve the complexity bound for counting integer points in general rational polytopes. We also improve the bound for polyhedra of bounded codimension; this includes the counting version of the \emph{Unbounded Subset-Sum} problem as a special case.