Relative entropy is a fundamental class of distances between probability distributions, with widespread applications in probability theory, statistics, and machine learning. In this work, we study relative entropy from a categorical perspective, viewing it as a quantitative enrichment of categories of stochastic matrices. We consider two natural monoidal structures on stochastic matrices, given by the Kronecker product and the direct sum. Our main results are complete axiomatisations of Kullback-Leibler divergence and, more generally, of Rényi divergences of arbitrary order, for each such structure. Our axiomatic theories are formulated within the framework of quantitative monoidal algebra, using a graphical language of string diagrams enriched with quantitative equations.
Autonomous vehicles (AVs) couple safety-critical perception to thermally constrained compute. Physical adversaries can spoof thermal infrared (IR) sensors used for night driving, while unprivileged software co-tenants can fingerprint perception workloads from thermal, dynamic voltage and frequency scaling (DVFS), and thermal event interrupt (TEI) telemetry required for package safety. These paths differ in mechanism and privilege, but both exploit interfaces that AV stacks cannot disable without compromising Safety of the Intended Functionality (SOTIF). Existing microarchitectural and CPS surveys cover cache timing, adversarial ML, and camera or LiDAR spoofing; none frames physical thermal IR attacks and unprivileged silicon thermal telemetry as one residual-risk class. This survey systematizes thirteen works into thermal perception integrity, telemetry leakage, and model-information extraction, comparing them by privilege, timescale, co-tenancy, deployability, and AV relevance while mapping defenses to residual risk. A commodity proxy supports the software co-tenant branch only; physical IR evidence comes from cited literature. We identify open problems in on-target automotive evaluation, fusion trust under IR spoofing, chiplet thermal coupling, capability-based telemetry, and standards-ready threat analysis, motivating co-design across sensors, operating systems, silicon, and automotive standards.
It is a well-known theorem of homotopy type theory, originally due to Voevodsky, that function extensionality holds inside any univalent universe. We consider a weaker variant of the univalence axiom, asserting that the wild category formed by the universe is univalent, which we call categorical univalence. We show that categorical univalence does not imply function extensionality by an analysis of Von Glehn's polynomial model construction, which produces models of Martin-Löf type theory that always refute function extensionality. We find in particular that when the base model has a univalent universe, its polynomial model has a universe that is categorically univalent but lacks function extensionality.
Large language models can generate fluent explanations, but effective tutoring requires supporting the learner's thought process, not just delivering content. Metacognitive tutoring targets this gap by prompting planning, monitoring, debugging, and evaluation, and, crucially, deciding when to be active versus minimally present, based on learner signals and trajectory. We introduce MetaCLASS, a learning-science grounded framework that formulates metacognitive tutoring as move selection over 11 interpretable actions aligned to self-regulated learning (SRL) processes. MetaCLASS uses a two-phase procedure that first plans a pedagogical trajectory conditioned on learner profiles (calibration, help-seeking) and then generates natural dialogue consistent with that plan. This process yields 1,015 conversations spanning 7,711 turns, each annotated with turn-level metacognitive labels and supported by comprehensive validation analyses. We benchmark nine open-source LLMs on predicting the next coach move given a question and dialogue context. The best model achieves only 43.2% accuracy in coach move prediction, and models exhibit compulsive intervention bias: in turns where effective metacognitive tutoring requires silence (41.7% of cases), models predict "no intervention'' only 6.6% of the time, while severely over-predicting high-intervention moves. These results show that traditional content-based tutoring ability does not translate to metacognitive tutoring competence, positioning MetaCLASS as a testbed for developing tutors that promote SRL.
Hao Wu, Jiyu Zhu, Amir Kafshdar Goharshady, Jie An, Bican Xia, Naijun Zhan
原文摘要
In this paper, we address the complexity barrier inherent in Fourier-Motzkin elimination (FME) and cylindrical algebraic decomposition (CAD) when eliminating a block of (existential) quantifiers. To mitigate this, we propose exploiting structural sparsity in the variable dependency graph of quantified formulas. Utilizing tools from parameterized algorithms, we investigate the role of treewidth, a parameter that measures the graph's tree-likeness, in the process of quantifier elimination. A novel dynamic programming framework, structured over a tree decomposition of the dependency graph, is developed for applying FME and CAD, and is also extensible to general quantifier elimination procedures. Crucially, we prove that when the treewidth is a constant, the framework achieves a significant exponential complexity improvement for both FME and CAD, reducing the worst-case complexity bound from doubly exponential to single exponential. Preliminary experiments on sparse linear real arithmetic (LRA) and nonlinear real arithmetic (NRA) benchmarks confirm that our algorithm outperforms the existing popular heuristic-based approaches on instances exhibiting low treewidth.
Reinforcement learning environments are now a primary lever for improving large language model (LLM) capabilities in post-training, yet most agentic benchmarks remain static: the world moves only when the agent acts, the reward is a terminal verdict, and the pass bar is set arbitrarily. We introduce StoreBench, a live-commerce environment in which an agent runs a mid-size online apparel store on a production-grade commerce backend, testing long-horizon planning and economic judgment under uncertainty. Customers order around the clock, suppliers reprice and fail, and market shocks arrive with partial or no warning. The agent acts through the same 29 merchant tools a human operator would use, under a windowed operation budget that makes simulated time a function of actions taken, so model latency cannot influence simulated time. Pass thresholds are calibrated against scripted anchor policies, the reward is hardened against a catalogue of reward hacks, and every episode replays identically given a sequence of actions. We evaluate seven frontier LLMs on 11 scenarios of 30 to 45 days and a full simulated year, over three world seeds at matched reasoning effort. No model matches the scripted smart-triage policy on average: the best, DeepSeek-V4-Pro, passes 49% of task-seed cells against the heuristic's 97%. Human experts working through the same tools and budgets outscore every model (mean composite 0.708 vs. 0.700). Over a full simulated year under the Claude Code harness, most models show dramatic performance improvement. In a GRPO post-training run, Qwen3.5-27B trained on only five disjoint tasks raises its mean composite on the held-out evaluation tasks from 0.136 to 0.373. We release five example training-split tasks, ten sample trajectories, and the scoring and verification tooling; the full environment and evaluation suite are withheld to keep the benchmark uncontaminated.
This paper presents an online trajectory and morphology planning framework for morphing quadrotors to traverse passages narrower than their unfolded width using only a 2D LiDAR for obstacle sensing. Fixed-frame vehicles cannot pass through such gaps, and classical artificial potential field (APF) based obstacle costs seal them off for planners as well, because the repulsive cost grows sharply inside a tight gap. The core enabler of our framework is a novel cost reduction factor, computed in closed form from the LiDAR data, that cancels the obstacle cost along the medial axis of a narrow passage. A true opening thus becomes a low-cost corridor, while the full repulsive cost is retained everywhere else. The cost reduction factor is embedded at every prediction node of an acados-based nonlinear MPC, which jointly plans the trajectory, yaw, and fold angle of the platform. A user-specified minimum-width limit ensures that the cost reduction factor only opens the corridors that the platform can actually pass. Gazebo/PX4 simulations and hardware experiments on the MorphoCopter demonstrate autonomous folding and unfolding through curved, inclined, and cluttered passages narrower than the unfolded width, where the standard repulsive cost fails to pass safely in most of the trials. The implementation code is available at \href{https://github.com/harshjmodi1996/morphocopter_mpc}{\textcolor{blue}{Github}} and a short video about the paper is available at \href{https://zh.engr.tamu.edu/wp-content/uploads/sites/310/2026/10/Morphocopter_planning_final.mp4}{\textcolor{blue}{\textbf{video link}}}.
There is a huge gap between numerous intriguing applications fostered by on-device large language model (LLM) fine-tuning (FT) from fresh mobile data and the limited resources of a mobile device. While existing server-assisted methods (e.g., split learning or side-tuning) may enable LLM FT on the local mobile device, they suffer from heavy communication burdens of activation transmissions. In addition, their multiple on-device forward passes over the same samples across training epochs lead to substantial iterative computation on the mobile device. To address those issues, we develop Eco-MobiLLM, an efficient LLM FT method which can be deployed on the mobile device via server-assisted side-tuning. To reduce communication cost, Eco-MobiLLM develops an activation shortcut that transmits only the tokens involved in the loss calculation instead of full activation matrices to guide the side network tuning. Besides, to improve computing efficiency and further accelerate FT convergence, Eco-MobiLLM introduces a novel two-stage training paradigm, which allows the server to reuse historical activations and saves the mobile device from repeatedly computing forward passes for the recurring data samples. Extensive experimental results demonstrate Eco-MobiLLM's superiority.
The standard method of exponential moments for proving concentration bounds can often be replaced by an argument based on elementary symmetric polynomials. We introduce an additional element of randomness into this framework, which reduces the problem to bounding product moments over a uniformly sampled set of indices. We show that this approach gives useful bounds in three settings. For read-$Δ$ families under limited independence, we obtain bounds governed by the degrees of randomly induced dependency subgraphs, improving the dependence on worst-case degrees. For random linear hashing over arbitrary finite fields with (semi-)random inputs, we derive fixed-bin and maximum-load bounds by controlling the rank defect of random tuples of input keys. Finally, for stochastic processes, we show how decay of product moments yields concentration bounds, recovering the spectral and mixing-time scales for finite-state Markov chains.
Mechanisation of a mathematical definition (also referred to as formalisation) has many benefits. Here, we focus on mechanising denotational semantic definitions of programming languages by embedding them in the Agda language. The Agda type-checker detects and reports any issues with the wellformedness and type correctness of the embedded definitions. To minimise the effort required, and to facilitate correlation of the original definition with its Agda embedding, mechanisation should not involve significant reformulation or extension. Here, we show how to embed conventional Scott-Strachey denotational definitions in Agda with almost no changes to $λ$-notation or domain equations. Agda notation for definitions of types and functions corresponds closely to the conventional meta-notation of denotational semantics. We have developed a collection of Agda modules with postulated types for commonly used domain constructors and their associated operations. We illustrate our approach with mechanisations of three denotational definitions: a model of the untyped $λ$-calculus, Plotkin's denotational semantics of PCF, and a semantics of a sublanguage of Scheme. In previous work, similar mechanisations in Agda have revealed several unsuspected wellformedness issues in published denotational definitions. Some of the postulated properties of our definitions are inconsistent with a classical set-theoretic interpretation of Agda. We conjecture that they would be consistent with an interpretation of Agda in a higher-order intuititionistic logic, such as that used by Simpson in his work on synthetic domain theory. After recalling previous work on mechanising domain theory, we consider the possibility of mechanising synthetic domain theory in Agda.
Sixing Chen, Ji-An Li, Saner Cakir, Sinan Akcali, Kayla Lee, Marcelo G. Mattar
原文摘要
Large language models (LLMs), especially reasoning models, generate extended chain-of-thought (CoT) reasoning that often contains explicit deliberation over possible futures. Yet it remains unclear whether this deliberation constitutes genuine planning, how it is structured, and what aspects of it drive performance. Here we introduce a method for characterizing LLM planning by extracting search trees from reasoning traces in the board game four-in-a-row. By fitting computational models to the extracted search trees, we quantify how plans are structured and how they influence move decisions. We find that LLMs search less deeply than humans, and that their performance is better predicted by search breadth than depth. Most strikingly, although LLMs expand deep nodes in their traces, their move choices are best explained by a myopic model that ignores those nodes entirely. A causal intervention that selectively prunes CoT paragraphs supports that move selection depends predominantly on shallow rather than deep nodes. These patterns contrast with human planning, where performance is driven primarily by deep search. Together, our findings reveal a key difference between LLM and human planning: human expertise rests on deeper search, whereas LLMs generate deep lookahead but fail to act on it. This dissociation offers concrete targets for aligning LLM and human planning. More broadly, our framework provides a generalizable approach for interpreting the structure of LLM planning across strategic domains.
In Polyamorous Scheduling Problem, we are given an edge-weighted graph and must find a periodic schedule of matchings in this graph which minimizes the maximal weighted waiting time between consecutive occurrences of the same edge. This NP-hard problem generalises Bamboo Garden Trimming and is motivated by the need to find schedules of pairwise meetings in a complex social group. We present a $4 G^*$ algorithm for Polyamorous Scheduling, improving the previously known bound of $3 + \sqrt{5} \approx 5.236$. Our algorithm is inspired by the Deadline-Driven Heuristic which is optimal for Bamboo Garden Trimming (BGT).
We study estimation and detection of high-order cumulant tensors from $n$ i.i.d. observations of a $p$-dimensional random vector, with performance measured in the tensor spectral norm. Under sub-Gaussianity, we show that the minimax rate for estimating the order-$d$ cumulant tensor is $\sqrt{p/n}\wedge 1$. In contrast to covariance estimation, the natural plug-in estimator obtained by replacing population moments in the moment--cumulant formula with sample moments, which we call the sample cumulant tensor, is generally not rate-optimal for $d\ge 3$, and we construct an estimator that attains the minimax rate. On the computational side, we study testing whether the $d$-th order cumulant tensor vanishes after whitening. We provide evidence, under a model-specific low-degree conjecture, that detection is computationally hard when $n\ll p^{d/2}$. At the same time, we identify a regime in which an efficiently computable estimator achieves error smaller than the separation at which polynomial-time tests can reliably distinguish the null from the alternative under the same conjecture. This reveals an unusual reverse detection--estimation gap: computationally efficient detection can be harder than computationally efficient estimation. The underlying reason is that the relevant loss, tensor spectral norm, is itself NP-hard to compute, creating a new form of computational--statistical gap.
Type systems which account for resource sensitive computations can generally be split into two styles: First, substructural logics such as Linear Logic which seek to restrict weakening and contraction and reintroduce them in a controlled manner; And second, graded systems which allow weakening and contraction by default, but track the use of variables quantitatively in some algebraic structure -- usually a semiring. We present GRASS (Graded and substructural), a type system which incorporates mechanisms from both of these approaches, thus allowing maximally flexible control over variable usage. Furthermore, GRASS allows grades from an arbitrary collection of grade algebras to coexist in the same system, thus allowing different variables to be controlled with respect to different notions of resource within the same program. We develop the categorical semantics of GRASS, and find that, on the level of categorical semantics, it subsumes multiple established systems such as LNL, Adjoint Logic, and mGL.
We refute the Strong Roberson Conjecture, which asserts that adding any graph outside a graph class closed under minors and disjoint unions strictly increases the distinguishing power of homomorphism counts from that class. More precisely, we construct connected graphs $H$ for which homomorphism counts from graphs excluding $H$ as a minor determine the number of homomorphisms from $H$ to any target graph. We also refute the analogous conjecture with immersions in place of minors. The proof introduces a method for deriving exact homomorphism count dependence from modular equivalences. We obtain these modular equivalences for infinitely many primes using prime-order automorphisms and the corresponding orbit quotients. We complete the refutations by proving that suitable circulant graphs with two jump lengths and related cubic bipartite graphs exclude their corresponding orbit quotients as minors and immersions, respectively.
When a transformer is trained from scratch on corpora that solve the same mathematical problem both correctly and incorrectly, which solution does it prefer? The usual test is pairwise, and it is not neutral: when the error can take several forms, a model that merely reproduces training frequencies wins it without preferring anything. We instead score the correct completion against every string the error generator can produce, a share on which frequency matching scores exactly 0.500. At 86M parameters the correct completion holds 59.0% +/- 0.8% of the mass against random errors (59.4% on an independent corpus), where the single realized error gives 77.6% and pair accuracy 87.4%, a three- to fourfold overstatement of the excess above chance; below 86M the primary corpus shows no preference. The coherent false rules of the main corpora are not filtered at any size (43.5--48.8%; the shortfall below 0.5 is a completion-length effect). Two experiments built to identify a compression mechanism raise the pairwise measures while the summed share stays flat or falls: both are negative controls, and compression remains motivation, not a measured mechanism. With the scored false strings held fixed, a false rule whose selector the model can learn moves the correct share from 5 points above the null to 7 below (418M--3.08B), though the correct completion still outweighs the selected string. What these models leave unfiltered is consistent or learnable false structure (one single-rule corruption excepted), not falsehood as such.
Amir Arjomand, Amin Boudesh, Farnoush Bayatmakou, Kenneth B. Kent, Arash Mohammadi
原文摘要
Recent statistics indicate that approximately 1.3 billion individuals worldwide suffer from hypertension, a leading cause of premature death globally. Blood Pressure (BP) serves as a critical health indicator for accurate and timely diagnosis and/or treatment of hypertension. Traditional BP measurement methods rely on cuff-based approaches, which lack real-time, continuous, and reliable BP estimates, crucial for the timely diagnosis/treatment of hypertension. Driven by recent advancements in Artificial Intelligence (AI) and Deep Neural Networks (DNNs), there has been a surge of interest in developing data-driven and cuff-less BP estimation solutions. In this context, current literature predominantly focuses on coupling Electrocardiography (ECG) and Photoplethysmography (PPG) sensors, though this approach is constrained by reliance on multiple sensor types. An alternative, utilizing standalone PPG signals, presents challenges due to the absence of auxiliary sensors (ECG), requiring the use of morphological features while addressing motion artifacts and high-frequency noise. To address these issues, the paper introduces the TransfoRhythm framework, a Transformer-based DNN architecture built upon the recently released physiological database, MIMIC-IV. Leveraging the Multi-Head Attention (MHA) mechanism, TransfoRhythm identifies dependencies and similarities across data segments, forming a robust framework for cuff-less BP estimation solely using PPG signals. To our knowledge, this paper represents the first study to apply the MIMIC IV dataset for cuff-less BP estimation. TransfoRhythm achieves highly accurate results with a Root Mean Square Error (RMSE) of [2.21, 1.84] and a Mean Absolute Error (MAE) of [1.37, 1.06] for systolic and diastolic blood pressures, respectively.
A fine-tuned Vision-Language-Action (VLA) policy can easily place the alphabet soup in the basket on command, just as it saw during fine-tuning. However, if the basket is shifted five centimeters to the left, it drops the soup off the table. We developed an interpretability pipeline to investigate what such a policy is computing. Injecting task A's activations into the action expert while the policy looks at task B's scene makes $π_{0.5}$ execute task A's motor trajectory in 99.6\% of episodes ($n{=}1{,}968$). The injected program is bound to absolute workspace coordinates and ignores the visible scene, which provides a mechanistic account of the perturbation brittleness that concurrent benchmarks report. The binding holds on a physical UR10e: directed replay redirects the commanded task in 26/26 rollouts in one direction and 18/26 in the other, with no-injection and random-direction controls at the floor, and once the source object is moved 10-15 cm the arm grasps and releases at the recorded coordinates in 50/50 trials, picking up the wrong block when one sits there and closing on empty air when none does. Applying the same interventions to five VLAs on LIBERO, MetaWorld, SimplerEnv, and ALOHA over $420{,}000+$ rollouts yields three further findings: every architecture encodes the instruction, and four of the five weight it below vision whenever the scene alone identifies the goal; under the standard replacement write-back the SAE pooling that preserves behavior is architecture-specific ($π_{0.5}$ per-token, SmolVLA indifferent), and goal selection and motor generation separate into VLM and expert pathways wherever the two are distinguishable. We release \textbf{Action Atlas} (https://action-atlas.com), an interactive platform covering all 520+ trained SAEs, 79 identified concepts, and the scored UR10e rollouts. Code: https://cwru-aism.github.io/vla-interp-page/
Xin Xu, Ke Cheng, Hao Wang, Yangru Huang, Yanghao Xiao, Zhichao Chen, Zhisheng Zhong, Xun Liu 等 14 位作者
原文摘要
Improving an agent through successive revisions requires balancing exploration and evaluation under a limited budget. A narrow search can miss useful sequences of edits, while fully evaluating every intermediate version often exhausts the budget before those sequences yield a verified improvement. Inspired by the draft--verify principle of speculative decoding, we propose Speculative Recursive Self-Improvement (SpecRSI), a two-level framework coupling inner-loop agent search with outer-loop improver learning. SpecRSI drafts successors before their predecessors are fully evaluated, while adopting only fully evaluated, admissible versions. In the inner loop, Verification-Aware Tree Search (VATS) allocates the budget between further edits and outstanding checks using the predicted verified return of a continuation. In the outer loop, Group Verification Optimization (GVO) updates the improver from completed searches while keeping the real evaluator fixed. We evaluate SpecRSI on scientific optimization (including active queue management (AQM) for network infrastructure), coding, and general agent-system benchmarks. Under matched evaluation-call budgets, SpecRSI improves scientific terminal gain over Janus by 60.0%--96.3% and area under the budget curve by 32.1%--91.6%, while achieving higher means across all 50 scientific backbone--task pairs and outperforming MetaRSI-v1 by 1.30--2.70 percentage points (pp) in system score across twenty backbones. On instance-level comparisons, it further improves PyBaMM terminal gain by 142.6% and SWE-Pro success by 6.0 pp at 7.0% and 9.2% lower token cost, respectively. These results demonstrate the efficacy of the proposed approach.
Kexin Ding, Yang Zhou, Can Jin, Feng Tong, Mu Zhou, Dimitris N. Metaxas
原文摘要
The growth of agent skills has transformed how agentic systems are built, evaluated, and deployed. As skill libraries continue to scale, rigorous evaluation becomes critical to ensuring their utility, quality, and safety in real-world applications. Consequently, the field is undergoing an emerging paradigm shift from isolated skill creation to automated, evaluation-driven skill evolution. In this survey, we systematically examine the landscape of skill evolution and evaluation beyond foundational skill creation. We categorize evolution into four distinct paradigms, spanning execution feedback, trajectory distillation, compression, and reinforcement learning, showing how each element contributes to improving skill utility and reliability. We also provide an analysis of six skill-centric benchmark categories, identifying structural gaps in benchmark coverage, trade-offs, and metric richness to advance skill research. Finally, we identify open directions for building skill ecosystems that are generalizable, efficient, and verifiably safe. The project URL is https://github.com/Cassie07/AgentSkill_Survey
Siri Expressive Voices synthesize rich, configurable speech in real time and entirely on device, powered by AFM 3 Core Advanced, Apple's most powerful on-device foundation model. This work presents the memory-efficient audio synthesis architecture behind that capability: a detokenizer that converts the semantic audio tokens emitted by the foundation model into high-fidelity audio within the tight compute and memory budget of the Scalable Matrix Extension (SME). We convert semantic audio tokens to a residual vector quantization (RVQ) representation with a three-component design, a streaming encoder, a temporal decoder, and a depth decoder, that systematically decouples temporal and depth processing. A single reusable depth decoder with Diffusion Transformer (DiT)-style stage conditioning generates all RVQ levels autoregressively, replacing the dedicated per-level decoders of prior multi-decoder architectures, while causal sliding window attention with fixed-window key-value caching yields constant memory complexity independent of sequence length. Deployed on the AMX, the detokenizer sustains roughly 10 ms per generation step, about 16x faster than real time, with a peak runtime memory of only 21 MB and 329 MB of on-device assets, enabling continuous streaming synthesis of 20-320 seconds of audio. This constant, small footprint replaces the linear and quadratic memory scaling of conventional transformer- and GAN-based approaches. Ablation studies validate the key architectural components, and audio quality assessment confirms that the architecture maintains synthesis fidelity while achieving efficiency gains over existing methods. Operating at a 1-billion-parameter activation size within AFM 3 Core Advanced, it improves Mean Opinion Score by +0.28 overall (4.15 vs. 3.87) and by +0.42 on conversational speech (4.24 vs. 3.82) over the prior on-device text-to-speech system.
Changes to what a robot can see do not mean that there is a change in the problem it is trying to solve, and many language-conditioned manipulation systems struggle with this distinction. When something in a scene moves, the robot might need to execute its actions in different places, but its manipulation plan can stay the same. We propose Sequential Planning via Anchored Robotic Keypoints (SPARK), which describes a task as a symbolic behavior tree that we anchor to the current 3D observations when each action is called. By verifying the plan at runtime we can recover from failures locally and anchor the objects to the scene again. We ask for a new plan only if this fails. We isolate this concept by taking a fixed plan and moving objects in the scene while executing. When displacing objects by $10$\,cm we were still able to complete $66.7\%$ of trials with scene-diff verification and layered recovery, and we did not have to change the plan. Without practicing on the tasks being tested or using ground-truth object poses from the simulator during rollouts, we test our full system (with one open-weight 27B planner with 4-bit weights) on LIBERO-PRO. Under this protocol SPARK achieves a mean success of $75.1\%$ across the 6 position and task settings and $69.7\%$ on the held-out language and object settings. We also achieve a $68\%$ success rate across 11 physical task-embodiment settings on 3 different robots. Our work demonstrates that for an important class of failures, the robot does not need to ask for a new plan for its task. It can be enough to update the system's view of the world. Code and videos can be found at https://cwru-aism.github.io/spark-page/
Conventional federated learning (FL) relies on parameter averaging, which forces clients to be doubly homogeneous: it demands an identical architecture and degrades under non-IID data. Real-world deployments usually break both assumptions. We sidestep both by building a decentralized knowledge distillation framework in which each client evaluates its peers' model snapshots on its own local data and distills from the resulting soft predictions. Because knowledge is transferred through the shared class posterior, clients are free to run different architectures; and because every teacher is evaluated on the student's own device, raw data never leaves the client, with no central server or public dataset required. Within this setting, we identify and address an under-examined problem: how to combine the peer teacher predictions. Existing methods, like uniform averaging, ignore how knowledge reliability varies across teachers and classes. We propose Class-wise Reliability-Aware Distillation (CRAD), which, per class, first discards teachers that disagree with the peer consensus and then takes a weighted average of the rest, weighting each teacher by its per-class reliability (precision, or inverse variance). Since the variance of an accuracy from $n$ samples scales as $1/n$, support enters automatically: among the teachers that survive filtering, a teacher is trusted for a class to the degree that it is both accurate and well-evidenced for it. On CIFAR-10, CIFAR-100, and PathMNIST, across heterogeneous architectures under severe non-IID skew, CRAD consistently outperforms competing methods in global accuracy. Our code is publicly available at https://github.com/bbilbeis/CRAD.
Although structured workflows empower Large Language Models (LLMs) to tackle complex problems, automating their creation is severely hindered by a vast combinatorial search space, frequently resulting in inflexible and resource-heavy offline training dependencies. To address this, we conceptualize workflow generation as an intertwined topology-and-execution search paradigm, where the broader topological layer dictates subtask boundaries and lower-level execution outcomes actively reshape the topology itself. Building on this foundation, we introduce HierFlow, a training-free, test-time hierarchical search architecture that automates agentic workflow design by merging feedback-guided topology adjustments with a fast, MCTS-inspired tree search for sub-workflow optimization. HierFlow maximizes efficiency through an intelligent gating module that selectively triggers execution-level searches based on contextual necessity, a mechanism we further support with an in-depth analysis detailing how varying degrees of cross-task coupling impact the effectiveness of hierarchical splitting. Comprehensive testing across question answering, mathematical reasoning, and code generation benchmarks confirms that HierFlow consistently outperforms strong baselines, delivering an optimal balance of high-quality results and computational efficiency without any additional training overhead.
Francisco Valentini, Edgar Altszyler, Martin Fajcik
原文摘要
Neural retrievers are trained to estimate query-document relevance from annotated query-document pairs. Yet annotation protocols may not purely reflect relevance: they select only a subset of documents for labeling, and this selection can favor certain document types over others. We investigate whether supervised bi-encoder retrievers implicitly learn a document-level relevance prior: a query-independent signal encoded in their representation space as a side effect of training on annotated data. We estimate this prior by training simple classifiers on frozen document embeddings and evaluate three state-of-the-art retrievers across multiple IR benchmarks. We find that supervised neural retrievers encode relevance priors that generalize to unseen documents and are consistent across models. These priors are associated with a findability gap: documents with lower prior are systematically harder to retrieve, even when genuinely relevant. This effect appears in supervised dense retrievers but is weaker and less consistent in BM25, and it persists under controlled matched-document comparisons. Using LLM-based explanations, we find that judged-relevant documents tend to be comprehensive, self-contained summaries of mainstream topics, while niche, fragmentary, or highly technical content is often left unjudged. Retrievers appear to internalize this bias, ranking documents with these favored features higher than documents that lack them, independently of their actual relevance. Our findings expose a structural limitation of supervised retrieval: models trained on annotated data do not just learn relevance, but also the implicit document preferences in their training data.
Maciej Satkiewicz, Roberto Corizzo, Marcin Pietroń
原文摘要
In linear models, visualizing a weight vector naturally reveals the model's preferred input direction, but extending this intuition to deep networks via gradients or GradientAscent often yields brittle or adversarial-looking features. We argue that deep networks are better understood as input-conditioned affine operators, whose natural adjoint action pulls a neuron's preferred direction back to input space. We further refine this representation through optional backward-only softening for hard-gated layers and propose pullbacks as an alternative backward operator for existing explanation procedures, including local attribution, smoothing, and iterative ascent. Across convolutional architectures (ResNet50, VGG) and transformer-based models (PVT), replacing gradients with pullbacks yields improvements in 46 out of 60 comparisons across five faithfulness, stability, and localisation metrics. In particular, PullbackAscent improves mean scores over GradientAscent across all five primary metrics on each architecture, reducing mean Infidelity by factors of 2.8 to 5.5. Strikingly, it also produces sharp, perceptually aligned and target-specific perturbations in only a few steps, without post-processing or explicit perceptual priors. More broadly, pullback-based methods achieve the best mean in 13 out of 15 architecture-metric combinations, outperforming established attribution baselines. Our results suggest that adjoint transport can serve as a reusable backward primitive, enhancing existing explanation procedures without architectural changes or elaborate attribution pipelines. We use "Semantic Pullbacks" (SP) to denote the resulting family of pullback-based explanations.
Reachability analysis is central to safety-critical control, robotics, and neural network verification, but classical computational methods, such as Hamilton--Jacobi reachability and set propagation, scale poorly with state dimension. Sampling-based methods have emerged as a promising alternative, often providing finite-sample guarantees that bound the probability-mass left uncovered. However, an explicit account of how the geometry of the initial set, the dynamics, and the sampling law affect the accuracy of the estimator is not fully available in the literature. We study this by casting sampling-based reachable-set recovery as geometric support estimation over a family of problems specified by an initial set, its dynamics, and a sampling law. First, we identify two regularity properties, positive reach of the initial set's complement and Lipschitz continuity of the dynamics, that together make recovery well-posed: a probability-mass coverage guarantee can be upgraded to accuracy $r$ in Hausdorff distance. Second, we bound the resulting sample complexity: recovery is achievable with $\tilde{\mathcal{O}}\big((e^{LT}/r)^n\big)$ samples, exponential in both the state dimension and the time horizon. Third, we show that neither can be removed: a minimax lower bound of $\tildeΩ\big((e^{LT}/r)^n\big)$ holds for every estimator, so the exponential dependence on dimension and the degradation over the horizon are both intrinsic, not artifacts of a particular method. Experiments on nonlinear systems confirm that adversarial sampling improves constants but not the scaling.
Tensor-train (TT) compression reduces neural-network parameters, but the latency of a tensorized layer is set by its contraction order and its mapping onto hardware: a contraction path with fewer multiply-accumulate operations (MACs) is not necessarily faster on a systolic array. We present a deployment-constrained, latency-guided design space exploration (DSE) framework for fixed-rank tensorized neural networks (TNNs) under a fixed processing-element (PE) budget. Using MAC-guided path pruning, it evaluates feasible combinations of contraction paths, three PE-array partitions, and input-, output-, and weight-stationary dataflows. By minimizing the sum of inference and training latency estimates, it selects a single physical PE partition shared across layers and both modes, while allowing layer- and mode-specific paths and dataflows. On ResNet-18 and ViT-Ti/4, 25%-50% of the layers select a path other than the minimum-MAC one, the best dataflow varies across layers and models, and on a representative ViT layer the latency-selected candidate reduces simulated latency by 23.5% relative to the minimum-MAC candidate. Deployed on a Xilinx VU9P FPGA accelerator that supports both inference and training, the complete TT+INT8 system achieves 3.28x-4.00x inference and 3.42x-3.85x training speedups over dense implementations with the same PE budget.
Post-training LLM compression often relies on low-rank approximations, which force all columns of a projection matrix to share a single low-dimensional subspace. We propose CoSpaDi, a training-free compression framework that replaces this single-subspace assumption with a union-of-subspaces model via sparse dictionary learning. CoSpaDi factorizes each weight matrix into a dense dictionary and column-sparse coefficients, allowing different columns to select different subsets of dictionary atoms at the same storage budget. To preserve model behavior, we use calibration activations to transform functional reconstruction into a standard dictionary learning problem. Across LLaMA and Qwen models, CoSpaDi improves accuracy-compression and perplexity-compression trade-offs over SVD-based and structured pruning baselines at 20-40\% compression ratios, while naturally supporting sparse-dense computation and post-training quantization of sparse coefficients.
Omar A. Khan, Bao Han Trinh, Lee Lisle, Tiffany D. Do
原文摘要
Embodying non-human body parts in VR is a prevalent practice among certain subcultures and is a personally important creative outlet to many individuals. However, the discrepant morphology between real and virtual bodies can decrease Sense of Embodiment (SoE). Haptic feedback can compensate by increasing SoE felt towards non-human body parts, but there is a literature gap in comparing the effects of different haptic modalities, and their combinations, on SoE. Through an online survey sent out to social VR communities (n = 63), we determined that animal ears are a commonly embodied and ecologically valid non-human body part to study. We then ran a 2x2 within-subjects user study (n = 28) with two independent variables: active haptics, delivered through vibrotactile gloves, and passive haptics, delivered through a physical headband, for when participants reach up to touch virtual dog ears appended to their avatar in VR. Our findings show that (1) passive haptics produced the strongest overall embodiment outcomes, (2) combining modalities reduced the benefits of passive haptics, and (3) SoE towards non-human body parts positively correlates with SoE towards the entire avatar.