Wilson Wongso, Lihuan Li, Arian Prabowo, Xiachong Lin, Baiyu Chen, Hao Xue, Flora D. Salim
Generating high-fidelity synthetic Web of Things (WoT) telemetry, such as GPS trajectories, is increasingly important for transportation, urban planning, and location-aware services, particularly when privacy concerns restrict access to real-world mobility data. Existing trajectory generation methods face a trade-off between efficiency and road network fidelity: continuous-space methods generate trajectories efficiently but ignore road topology, while topology-aware methods rely on search-based autoregressive decoding that limits generation speed. We propose TrajDLM, a topology-aware trajectory generation framework based on block diffusion language models. TrajDLM represents trajectories as sequences of discrete road segments, combining a block diffusion backbone, graph-based road network representations, and topology-constrained sampling to generate realistic trajectories with topologically valid road transitions. To address the tendency of supervised models to favor near-shortest routes, we introduce TrajRFT, a rejection-sampling fine-tuning stage that improves route fidelity by training on self-generated routes selected for their similarity to real trajectories. Across three city-scale vehicle telemetry datasets, TrajDLM closely matches real-world mobility in population-level distributions, local route similarity, and route diversity, while generating trajectories up to $7.5\times$ faster than the strongest baseline. It also demonstrates zero-shot transfer to unseen transport modes. These results establish block diffusion with topology-constrained sampling and rejection-sampling fine-tuning as an efficient approach to generating realistic synthetic mobility telemetry for location-aware Web services and WoT applications.
Declan J. Curran, Sanaa Hobeichi, Hira Saleem, Hao Xue, Flora D. Salim
Downscaling is essential for generating the high-resolution climate data needed for local planning, but traditional methods remain computationally demanding. Recent years have seen impressive results from AI downscaling models, particularly diffusion models, which have attracted attention due to their ability to generate ensembles and overcome the smoothing problem common in other AI methods. However, these models typically remain computationally intensive. We introduce a Hierarchical Diffusion Downscaling (HDD) model, which introduces an easily-extensible hierarchical sampling process to the diffusion framework. A coarse-to-fine hierarchy is imposed via a simple downsampling scheme. HDD achieves competitive accuracy on ERA5 reanalysis datasets and CMIP6 models, significantly reducing computational load by running on up to half as many pixels with competitive results. Additionally, a single model trained at 0.25° resolution transfers seamlessly across multiple CMIP6 models with much coarser resolution. HDD thus offers a lightweight alternative for probabilistic climate downscaling, facilitating affordable large-ensemble high-resolution climate projections. See a full code implementation at: https://github.com/HDD-Hierarchical-Diffusion-Downscaling/HDD-Hierarchical-Diffusion-Downscaling.
Jiayi Li, Guancheng Zhou, Yisi Luo, Ruihua Chen, Penghao Jiang, Deyu Meng, Flora D. Salim, Piotr Koniusz
Large-scale PDE pre-training trains a single model on the data from multiple partial differential equations (PDEs). However, adding heterogeneous PDEs with different governing equations, physical variables, and variable interactions does not always improve predictive performance across tasks. Existing PDE mixture-of-experts models mitigate cross-task interference through learned routing, but typically use homogeneous expert architectures, making it difficult to directly reflect differences in state evolution across equations. We introduce HotPOT, an operator transformer with heterogeneous physics experts for large-scale PDE pre-training. HotPOT uses a shared encoder to capture information common across PDEs, while each PDE family uses a neural-numerical expert that pairs a learnable predictor with a fixed numerical update containing no trainable parameters. Each predictor outputs the quantities required by its paired numerical update, which then computes the next state. This design allows different PDEs to share a common encoder while retaining their own learnable predictors and numerical updates. Experiments across five PDE families and six settings show that HotPOT outperforms MoE-POT in all 24 matched-scale comparisons before and after fine-tuning, reducing rollout error by 50.7% on average. Notably, the 17M model also surpasses substantially larger MoE-POT models, and the performance gains extend to zero-shot prediction under unseen physical conditions. Code is available at https://github.com/cruiseresearchgroup/HotPOT-code.
Jiayi Li, Penghao Jiang, Baiyu Chen, Yifan Duan, Hira Saleem, Zhaonan Wang, Piotr Koniusz, Flora D. Salim
Neural operators efficiently simulate dynamical systems by repeatedly predicting the next state. However, small errors at each step can accumulate, causing long-term predictions to drift from the true trajectory. Long-term error reflects both the propagation of earlier prediction errors and new errors introduced at each step. How to reduce these errors is therefore a critical question. We propose the Spectral Generator Neural Operator (SGNO), which combines a linear estimate of the next state with a learned correction. The linear component uses consecutive states in the training data to fit a map from the current state to the next. A neural network corrects this prediction by learning changes that the linear model does not capture, including nonlinear interactions. Both components are trained jointly toward the same next-state target. Experiments across 13 periodic PDE configurations in one to three dimensions show that SGNO accurately learns state evolution on linear tasks and reduces 100-step rollout error by 13-74% over the strongest baseline under the main comparison protocol on each nonlinear task. Longer-horizon extrapolation tests show lower late-window prediction errors on several nonlinear tasks. These results show that combining fitted propagation with learned state-dependent correction can improve long-horizon autoregressive prediction on the evaluated PDE tasks. Code is available at https://github.com/cruiseresearchgroup/SGNO.
GlucoFM: A Dual-Stream Foundation Model for Continuous Glucose Monitoring
Zechen Li, Keerthana Natarajan, Weizhi Zhang, Menglian Zhou, Simon A. Lee, Yuwei Zhang, Maxwell A. Xu, Zeinab Esmaeilpour 等 14 位作者
AI 速览：提出轻量级CGM基础模型GlucoFM，将血糖动态分解为慢变趋势与短期偏差双流，并在大规模无标签CGM数据上预训练，用于代谢表型分类与餐后血糖响应预测。
Continuous glucose monitoring (CGM) provides a dense view of daily metabolic physiology, yet existing generic time-series and CGM-specific foundation models often encode glucose traces as entangled single-stream sequences, leaving their multiscale temporal structure only implicitly modeled. We present GlucoFM, a lightweight CGM foundation model that aligns irregular recordings to a 24-hour chronological grid, preserves observation masks, and decomposes glucose dynamics into slow-varying glycemic trend and short-term deviation streams. GlucoFM is pretrained on 109,066 hours of unlabeled CGM recordings from 477 subjects with masked contextual latent prediction over fused daily representations and temporal dynamics prediction over the two streams. Frozen pre-fusion probes confirm distinct temporal emphasis: state tokens preferentially preserve hourly glucose level, whereas event tokens better retain short-term change. Across four diverse cohorts and seven phenotype-classification tasks, GlucoFM achieves the strongest subject-disjoint linear-probing performance among evaluated baselines, improving average PR-AUC by 4.1 points over the best CGM-specific foundation model, while also supporting strong cross-dataset transfer and few-shot adaptation. Its gains are most pronounced on core metabolic outcomes, leading PR-AUC on all diabetes-risk and $β$-cell dysfunction tasks and on 3 of 4 insulin-resistance tasks. Beyond phenotyping, when combined with recent CGM, meal nutrition, and subject context, the same frozen encoder achieves the lowest two-hour postprandial glycemic response errors among evaluated methods for trajectory, incremental area under the curve, peak rise, and peak timing. Together, these results demonstrate that GlucoFM learns reusable frozen CGM representations spanning metabolic phenotyping and context-conditioned glucose-response prediction.
Yuncheng Hua, Sion Weatherhead University of New South Wales, Mehdi Jafari, Hao Xue The Hong Kong University of Science and Technology, Flora D. Salim
Automated simulator construction requires distributional fidelity, distinguishing it from generic code generation. We identify two failure modes in long-horizon LLM agents: contextual drift and optimization instability arising from conflating structural and parametric errors. We propose SOCIA-EVO, a dual-anchored evolutionary framework. SOCIA-EVO introduces: (1) a static blueprint to enforce empirical constraints; (2) a bi-level optimization to decouple structural refinement from parameter calibration; and (3) a self-curating Strategy Playbook that manages remedial hypotheses via Bayesian-weighted retrieval. By falsifying ineffective strategies through execution feedback, SOCIA-EVO achieves robust convergence, generating simulators that are statistically consistent with observational data. SOCIA-EVO’s code and data are available here: https://github.com/cruiseresearchgroup/SOCIA/tree/evo.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表
Chao Xue, Yao Wang, Mengqiao Liu, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu Beijing Information Science & Technology University, Chenyao Lu Tencent AI Lab 等 14 位作者
Supervised Fine-Tuning (SFT) is the standard approach for adapting large language models (LLMs) to downstream tasks. However, we observe a persistent failure mode: even after convergence, models often fail to correctly reproduce a subset of their own supervised training data. We refer to this behavior as the Incomplete Learning Phenomenon (ILP). This paper presents the first systematic study of ILP in LLM fine-tuning. We formalize ILP as post-training failure to internalize supervised instances and demonstrate its prevalence across multiple model families, domains, and datasets. Through controlled analyses, we identify five recurrent sources of incomplete learning: (1) missing prerequisite knowledge in the pre-trained model, (2) conflicts between SFT supervision and pre-training knowledge, (3) internal inconsistencies within SFT data, (4) left-side forgetting during sequential fine-tuning, and (5) insufficient optimization for rare or complex patterns. We introduce a diagnostic-first framework that maps unlearned samples to these causes using observable training and inference signals, and study several targeted mitigation strategies as causal interventions. Experiments on Qwen, LLaMA, and OLMo2 show that incomplete learning is widespread and heterogeneous, and that improvements in aggregate metrics can mask persistent unlearned subsets. The findings highlight the need for fine-grained diagnosis of what supervised fine-tuning fails to learn, and why.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表
Ghadir Alselwi, Hao Xue The Hong Kong University of Science and Technology, Shoaib Jameel University of Southampton, Basem Suleiman University of New South Wales, Flora D. Salim, Imran Razzak Mohamed bin Zayed University of Artificial Intelligence
Effective long-term memory management is crucial for language models handling extended contexts. We introduce the Enhanced Ranked Memory Augmented Retrieval ERMAR framework, which dynamically ranks memory entries based on relevance. Unlike prior models, ERMAR employs a novel relevance scoring mechanism and a pointwise re-ranking model for key-value embeddings, inspired by learning-to-rank techniques in information retrieval. By integrating historical usage patterns and adaptive retrieval, ERMAR achieves state-of-the-art results on standard benchmarks, demonstrating superior scalability and performance in long-context tasks.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表
Zechen Li Google, Baiyu Chen University of New South Wales, Hao Xue The Hong Kong University of Science and Technology, Flora D. Salim
Motion sensor time-series are central to Human Activity Recognition (HAR), yet conventional approaches are constrained to fixed activity sets and typically require costly parameter retraining to adapt to new behaviors. While Large Language Models (LLMs) offer promising open-set reasoning capabilities, applying them directly to numerical time-series often leads to hallucinations and weak grounding. To address this challenge, we propose ZARA (Zero-training Activity Reasoning Agents), a knowledge- and retrieval-augmented agentic framework for motion time-series reasoning in a training-free inference setting. Rather than relying on black-box projections, ZARA distills reference data into a statistically grounded textual knowledge base that transforms implicit signal patterns into verifiable natural-language priors. Guided by retrieved evidence, ZARA iteratively selects discriminative cues and performs grounded reasoning over candidate activities. Extensive experiments on eight benchmarks show that ZARA generalizes robustly to unseen subjects and across datasets, demonstrating strong transferability across heterogeneous sensor domains. These results mark a step toward trustworthy, plug-and-play motion understanding beyond dataset-specific artifacts. Our code is available at https://github.com/zechenli03/ZARA.
尚无 AI 解读，展示原文摘要
cs.CL · cs.AI
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表