ExRel 用可执行关系契约从真实题目合成可控、可验证的有向复用相关性监督数据,在 Medium 关系上优于规模匹配的自然数据与重放基线。
cs.IR
查看原文摘要
In K-12 education, question images combine text, mathematical expressions, diagrams, and diverse layouts. Variations in wording, conditions, answer options, or task scope can substantially alter their reuse utility despite visual similarity. We formulate this problem as directed reuse relevance, grading candidate utility as Strong, Medium, or Weak. However, generic similarity scores overlook task-specific reuse requirements, while natural supervision is dominated by Strong relations and lacks fine-grained coverage. We propose ExRel, an agentic, code-grounded framework for controllable and verifiable visual data synthesis. It reconstructs real questions into editable HTML/CSS seeds and compiles 15 relation subtypes into executable contracts specifying permitted edits, protected invariants, and query-candidate direction. Synthesized pairs undergo deterministic DOM/render validation and target-hidden, evidence-assisted review, with bounded repair before mild photographic degradation. Only rendered images are used for downstream training. Experiments show that ExRel supervision outperforms size-matched natural data and count- and class-matched replay, particularly on Medium relations, while maintaining comparable query-level ranking performance and improving robustness to visual degradation. Construction ablations demonstrate higher valid yield and fewer false admissions than less structured baselines. Human audits confirm 98.0% post-admission validity, while ExRel achieves a lower per-accepted-pair cost than outsourced annotation (0.70 vs. 1.10 RMB). Downstream answer routing further demonstrates the practical utility of the synthesized supervision.
We study community search when each member has its own tolerance for differences in position, as when members of a group interact repeatedly and influence one another. A latitude-consistent community (LCC) contains the query vertex, is connected, has minimum degree at least k, and bounds its diameter by the smallest member tolerance. The bound is exactly the condition under which every interaction in a connected closed group stays accepted under bounded-confidence dynamics for every choice of positive weights. It also prevents rejection under social-judgment dynamics, whereas the pairwise condition and other natural relaxations preserve neither guarantee. Finding a maximum LCC is NP-hard even in the plane, and with unbounded dimension, deciding whether an LCC exists is NP-complete for every fixed k>=2. LatSweep fixes the answer's smallest tolerance and turns the coupled bound into a best-first sweep over candidate thresholds, yielding an exact branch-and-bound algorithm, a (1,2)-bicriteria approximation and two feasible heuristics. Across eight graphs with up to 1.8 billion edges, six with synthetic positions and tolerances, the exact algorithm proves optimality on all but 9 of 3,700 efficiency queries; in a shared implementation on two large graphs, its branch-and-bound solver proves more queries than a reimplemented maximum-(k,r)-core algorithm, a mixed-integer program and CP-SAT. In simulations of closed groups, LCC groups keep every accepted interaction under uneven influence, whereas 3-40% of pairwise-compatible groups lose some. On three location-based networks, tolerance-agnostic groups often place another member's home beyond a member's estimated travel radius. Enforcing every member's tolerance causes no measurable loss of co-visits relative to spatial communities of the same size. Relative to the larger pairwise-compatible groups, it can cause a size-adjusted loss.
作者构建两个故意训练出特定错位行为的模型生物,形式化事后行为抑制问题,并提出 PEARL 方法以追求即时抑制、训练后持久性与通用能力保持的平衡。
cs.CL · cs.AI · cs.LG
查看原文摘要
Undesired model behaviors can emerge or re-emerge at any stage of training. Once observed, how effectively can we suppress them? To study this question, we first construct two model organisms--language models deliberately trained to exhibit a specific misalignment--that differ in what triggers the behavior. The narrow-trigger organism misbehaves only when the prompt contains a specific cue: it cheats on multiple-choice questions by following a planted hint that points to a wrong answer, and behaves normally otherwise. The broad-trigger organism has no such cue: after fine-tuning on insecure code, it gives harmful responses across a wide range of open-ended prompts. We then formalize post-hoc behavioral suppression and evaluate methods along three criteria: immediate suppression of the target behavior, durability under further training, and preservation of general capability. We compare standard methods--SFT, GRPO, and gradient-ascent unlearning--along these dimensions. Building on prior work on alignment shallowness, we hypothesize that more durable suppression requires the ability to recover from a wider range of misalignment states. We introduce Prefix-Expanded Adversarial Reinforcement Learning (PEARL), a GRPO variant that augments rollouts with continuations from intermediate states sampled along cached organism trajectories. Across both organism settings and two model families (Qwen3-4B, GPT-OSS-20B), PEARL achieves strong immediate suppression while preserving task accuracy, and remains durable under further fine-tuning on either the organism's data or unrelated data. It obtains the greatest benefit when the undesirable behavior is broad and cannot be addressed through targeted SFT. Together, our framework and method offer a step toward more durable post-hoc removal of undesired behaviors.
On-policy Self-distillation (OPSD) has proven effective for post-training large language models by constructing a teacher conditioned on privileged references. However, existing approaches keep the teacher fixed either throughout training by consistently exposing it to the full reference, or within each response by adaptively varying partial reference exposure only across responses. In this paper, we argue that neither of them is sufficient. Instead, every token needs its own teacher, as the teacher preference can vary token by token within the same response. Through a teacher-intervention pilot study, we validate that the teacher preference differs across tokens, and reveal that a preferred teacher can be identified for each token through majority voting and KL-based selection. Motivated by these findings, we propose Multi-teacher Adaptive On-policy Self-Distillation (MTA-OPSD), the first token-level adaptive OPSD method. Our method constructs a multi-teacher pool consisting of candidate teachers, and enables every token to adaptively select its preferred teacher during training. Experiments with Qwen3 models on mathematical reasoning tasks show that MTA-OPSD consistently outperforms existing OPSD approaches with an average gain of up to $+5.9$ over the base model, highlighting the importance of adapting teacher supervision at the token level. The code is available at https://github.com/xingzhejun/MTA-OPSD.
Large Language Model (LLM) agents, acting on their users' behalf to manipulate and analyze data, are likely to become the dominant workload for data systems in the future. When working with data, agents employ a high-throughput process of exploration and solution formulation for the given task, one we call agentic speculation. The sheer volume and inefficiencies of agentic speculation can pose challenges for present-day data systems. We argue that data systems need to adapt to more natively support agentic workloads. We take advantage of the characteristics of agentic speculation that we identify, i.e., scale, heterogeneity, redundancy, and steerability - to outline a number of new research opportunities for a new agent-first data systems architecture, ranging from new query interfaces, to new query processing techniques, to new agentic memory stores.
Transformer-based language models (LMs) are powerful and widely-applicable tools, but their usefulness is constrained by a finite context window and the expensive computational cost of processing long text documents. We propose to adapt pre-trained LMs into AutoCompressors. These language models are capable of compressing long contexts into compact summary vectors, which are then accessible to the model as soft prompts. Summary vectors are trained with an unsupervised objective, whereby long documents are processed in segments, and summary vectors from all previous segments are used in language modeling. We fine-tune OPT and Llama-2 models on sequences of up to 30,720 tokens and show that AutoCompressors can utilize long contexts to improve perplexity. We evaluate AutoCompressors on in-context learning by compressing task demonstrations and find that summary vectors are good substitutes for plain-text demonstrations, increasing accuracy while reducing inference costs. Finally, we explore the benefits of pre-computing summary vectors for large corpora by applying summary vectors to retrievalaugmented language modeling and a passage re-ranking task. Overall, AutoCompressors emerge as a simple and inexpensive solution to extend the context window of LMs while speeding up inference over long contexts.