跳到正文
研踪PaperTrail

论文库

检索计算机领域论文,按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+907新增论文
+744其中 AI 相关
+108已有论文修订
+94新增 AI 导读

arXiv 最近完成扫描:2026-11-19 06:59 · 全站最近入库:2026-11-19 06:59各信源进度 →

统计口径与数据新鲜度

统计 2026-11-18 11:54 — 2026-11-19 11:54(北京时间) · 点击数字查看对应简报;AI 相关包含在新增论文内。

最近完成检查:2026-11-19 06:59最近发现新增版本:2026-11-19 06:59库内论文最新更新时间:2026-11-18 08:00

每分钟刷新已保存的数据。空检查不代表采集异常;新收录可能包含历史论文。AI 导读按新增版本计数。

26,871 已收录论文22,375 AI 相关8,156 有修订112 HF 社区发现

浏览论文库

时间范围跟随排序:版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常,查看状态 →

找到 160 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-19 06:57(北京时间)
新论文 · v1arXiv:2611.04480

ReListen: Training-Free Evidence Localization in Hour-Long Audio with Frozen Audio Language Models

Ming-Hao Hsu, Shujie Liu, Yuxuan Hu, Yuxiang Wang, Yingda Shen, Jinyu Li, Zhizheng Wu

原文摘要

Audio language models are meant to answer questions from the audio itself, but an hour-long recording rarely fits in their context, and the answer often depends on a few seconds of it. Transcribing the recording first makes it fit but discards sounds and voices, while systems that return to the audio must be trained for this behavior. We propose ReListen, a training-free method in which a frozen audio language model finds its own evidence before answering. In a Jev-style scan, the model scores each 20-second window by the probability that it helps answer the question, and it then answers from the highest-scoring minute. This minute overlaps the annotated evidence far more often than transcript retrieval, for 88.5% of VoiceGiraffe questions. Without any change, the same scan also grounds events on LAT-Bench as well as a model trained for temporal grounding. Answering from it matches or exceeds hearing the whole recording with a small fraction of the audio, and on VoiceGiraffe it exceeds a transcript of the whole recording by 9.2 points. The selected evidence is also worth hearing rather than reading. Given the same excerpts, reading their transcript matches listening on speech content, but listening is clearly more accurate on questions about sounds and voices. Moreover, ReListen answers faster than every transcript-based system, and its memory stays constant even for recordings too long to hear whole on one GPU. Therefore, a frozen audio language model can search hour-long recordings by itself, without a transcript or additional training.

cs.CL · cs.SD · eess.AS

新论文 · v1arXiv:2611.03605

ufakzeka-kulak: Building and Evaluating a 31.6M-Parameter Turkish Speech Recognizer from Scratch

Sait Furkan Teke

原文摘要

ufakzeka-kulak is an open Turkish speech-to-text model of 31.6M parameters, a FastConformer encoder with a token-and-duration transducer (TDT) and a CTC head, trained from scratch and released under Apache-2.0. It writes cased, punctuated Turkish with numbers as words. Its 4,977 training hours come from sources whose licences allow commercial use; 281 hours are human-transcribed, and for the other 4,696 the training text is an uploader subtitle or a machine transcript, kept only where a second, independent reading of the audio agrees with it. The recipe, checkpoint, labelling and export choices were made on development data, and the test sets were scored once, in a release gate fixed before training. With greedy TDT decoding the normalised word error rate (WER) is 0.0921 (95 percent interval 0.0792 to 0.1055) on FLEURS Turkish test and 0.0916 (0.0873 to 0.0958) on Common Voice 27.0 Turkish test, lower than Whisper base and Whisper small on both sets with paired intervals wholly below zero; Whisper large-v3 (greedy) scores 0.0604 and 0.0767. On conversational speech the WER is 0.3807. We also report punctuation and casing, degraded audio, and how closely each shipped export reproduces the trained model.

cs.CL · eess.AS

新论文 · v1arXiv:2611.03769

Metadata-Conditioned Audio Transformers for Adaptive Respiratory Sound Classification

George Kontogiannis, Pantelis Tzamalis, Sotiris Nikoletseas

原文摘要

Respiratory sound classification from auscultation recordings is confounded by heterogeneous acquisition conditions, including recording device, auscultation site, and patient characteristics, that introduce systematic variability unrelated to the underlying pathology. Although such metadata is often available, existing approaches either ignore it or rely on naïve fusion strategies that do not model how distinct metadata sources influence different components of the acoustic representation. We propose a family of metadata-conditioning mechanisms for the Audio Spectrogram Transformer (AST), progressing from gated residual fusion to Feature-wise Linear Modulation (FiLM) adapted to the Transformer architecture. We further introduce two novel variants: Token-Aware FiLM (TAFiLM), which applies distinct modulation to classification and patch tokens, and Soft-Factorized FiLM (SoftFiLM), which learns soft masks to align metadata factors with disentangled feature subspaces. Experiments on the ICBHI benchmark demonstrate consistent gains across all conditioning strategies. SoftFiLM achieves 64.11% on the official 4-class task (+3.94% over the baseline) and 72.40% on the 2-class task, establishing a new state-of-the-art on the 2-class task and competitive performance on the 4-class benchmark.

cs.SD · cs.AI · eess.AS

新论文 · v1arXiv:2611.03077

Ananas: A Networked Audio System for Distributed Spatial Audio Applications

Thomas Rushton, Romain Michon, Tanguy Risset

原文摘要

Spatial audio has an accessibility problem. Largescale spatial and immersive audio installations remain costly and exclusive, with high barriers to entry. We propose ''Ananas,'' a networked audio system consisting of network protocols, control software and microcontroller firmware, with the aim of rendering spatial audio applications accessible to a broad cohort of potential users. Based on a network of low-cost microcontroller development boards programmed in the Faust programming language, synchronised to nanosecond order via IEEE 1588 Precision Time Protocol, and controlled by approachable, digital audio workstation-based tooling, Ananas represents an opportunity to disrupt and democratise the spatial audio landscape. In this paper we demonstrate Ananas's protocols for audio and control data transmission and device discovery, its methods for media clock and presentation time recovery, and its applications to wave field synthesis and higher order ambisonics in a modular, scalable form.

cs.SD · eess.AS

版本更新 · v2arXiv:2609.31948

Duplex-MPE: Benchmarking Multi-Party Interaction in Full-Duplex Dialogue

Chengqian Ma, Wenhao Feng, Weixuan Jin, Gaole Dai, Tianyu Xie, Yuexiao Ma, Zhaolu Kang, Xiangyu Zhao 等 10 位作者

原文摘要

Real-time full-duplex speech models can listen while speaking, enabling natural interaction without rigid turn boundaries. Existing benchmarks evaluate turn-taking, interruption handling and multi-round dialogue, but largely centre on a designated user rather than an assistant participating in a shared conversation among several people. We introduce Duplex-MPE to evaluate when such an assistant should answer, remain silent or stop speaking. The benchmark contains 2,000 scenarios with three or four human speakers and one assistant, each paired across explicit and implicit addressing of the same request. Models receive continuous conversation audio without transcripts or supplied turn boundaries. Four scores measure fresh response initiation, answer accuracy, silence preservation and stopping when a human resolves a request. We evaluate eight open-weight speech systems: MiniCPM-o 4.5, Moshi, FLM-Audio, Voila, Freeze-Omni, Realtime-Venus-Audio, Moshika-RL-Seamless and PersonaPlex. MiniCPM-o 4.5 leads on three scored capabilities, while frequent speech from other systems can coexist with inaccurate answers or failures to remain silent. A transcript-based Gemini 3.1 Pro reference responds 64.3 percentage points more often to explicit than implicit requests; paired tests detect no significant response-presence difference for the speech systems.

cs.SD · eess.AS

新论文 · v1arXiv:2611.02927

A Compact Statistical View of Musical-Instrument Directivity

Gioele Greco, Mirco Pezzoli, Raffaele Malvermi, Gian Marco Ricci, Fabio Antonacci

原文摘要

Directivity is a defining feature of musical-instrument radiation and is known to vary strongly with frequency. With the growing availability of measured directivity datasets and their use in sound-field simulation and spatial audio, there is a practical need for representations that are both compact and comparable across sources. This work addresses that need by proposing a statistically grounded description of frequency-dependent directivity that reduces complex radiation patterns to a small set of meaningful descriptors. The approach enables directivity patterns to be summarized in an interpretable way and compared consistently across frequency bands, even when the radiation is not well described by a single dominant direction. The resulting representation supports tasks such as analysis, categorization, and model building, providing a bridge between detailed measurements and higher-level descriptors useful for musical acoustics and audio rendering. Experiments on a corpus of 41 orchestral instruments show that the proposed method preserves the distance geometry well while reducing the full order-4, 30-band corpus from about 492~kB to about 100~kB in double-precision storage.

eess.AS · cs.SD · physics.class-ph

新论文 · v1arXiv:2611.02893

Voice-Native LLMs Hear Frustration but Do Not Listen to It

Hanjun Luo, Yuhan Lu, Hanan Salam

原文摘要

Users convey frustration in what they say and in how they say it, and voice-native models, which hear speech instead of an ASR transcript, can register both. Existing evaluations read unchanged performance as robustness without checking perception. Following Brunswik's lens model, we measure on identical audio whether frustration is expressed, perceived, and utilized. We cross frustrated wording with frustrated voice in a within-item 2x2 design over 353 tasks and release the 1,412 clips as VoxAffect. Across 8 models, outputs look robust, with no detectable vocal effect on accuracy for 7. With neutral wording, all 8 label the frustrated voice frustrated in at most 3.1% of clips, although 7 of them detect acted anger in 73-100% of clips and human listeners classify the voice correctly in 65.3% of judgments. Yet linear probes decode the frustrated voice from the audio encoders of the 2 open-weight models, with paired accuracy 0.67 and 0.69, and in proprietary models too the voice raises perceived frustration once the words agree. The models thus hear the frustrated voice but do not listen to it, whereas their invariance to frustrated wording, perceived in 40-75% of clips, is genuine robustness. Our work is openly accessible at https://github.com/Astarojth/hear-not-listen.

cs.SD · cs.AI · eess.AS

新论文 · v1arXiv:2611.02680

VioTech: Frame-Level Violin Playing Technique Detection with Multi-Scale Mixture-of-Experts

Ting-Kang Wang, Vincent K. M. Cheung, Nan-Tien Lai, Li Su

原文摘要

Playing technique is crucial for expressive violin performance, yet few violin technique detection models exist. Current systems mainly rely on note-level pipelines that first estimate note boundaries and then classify techniques. This is especially problematic for bowed string instruments such as the violin, where soft onsets and continuous bow-string interaction render note segmentation unreliable. Here, we propose VioTech, a frame-level violin technique detection framework that predicts techniques directly at 10 ms resolution without explicit note segmentation. To handle the large temporal-scale variation among techniques, VioTech combines a frozen MERT encoder with a multi-scale Mixture-of-Experts (MoE) architecture and task-specific routing for tonal, articulation, and legato-related predictions. We also introduce VioTech120, the first manually annotated dataset of violin playing techniques constructed from real solo performance recordings, as well as an accompanying note-labeling user interface. Results demonstrate that VioTech achieved consistent improvements over prior methods and significantly enhanced cross-dataset generalization, establishing a new state-of-the-art in violin technique-aware transcription.

cs.SD · cs.AI · cs.LG · cs.MM · eess.AS

新论文 · v1arXiv:2611.02628

ReplayFactor: Replay-Aware Dual-Attribute Learning for Practical Speech Deepfake Detection

Miao He, Jiayi Zhou, Yu Zheng, Zhongjie Ba, Peng Cheng, Yuxi Zhou, Junjie Zhu, Changhao Zhang 等 9 位作者

原文摘要

Advances in synthesis techniques have made deepfake speech increasingly realistic, amplifying voice security threats, including unauthorized voice cloning and impersonation. Although existing deepfake detectors perform well on digitally generated speech, their performance often degrades on physically replayed deepfakes, which constitute a low-cost and practical threat to real-world systems. Existing research focuses primarily on dataset construction and evaluation, while available dataset diversity and replay-aware detector design remain limited. We propose ReplayFactor, a dual-attribute framework that enhances replay evidence by fusing self-supervised learning (SSL) representations with an acoustic prior, while leveraging multi-task learning and supervised contrastive learning to promote robust, generalizable representations. We also construct ReplayFake-CN, a large-scale, multi-device Chinese dataset comprising approximately 185k replayed synthetic utterances to complement public benchmarks. Our framework improves macro-F1 from 60.99% to 76.44% and reduces EER from 11.86% to 3.05% on the EchoFake open set over the baseline, demonstrating strong potential for practical replayed deepfake detection.

cs.SD · eess.AS

新论文 · v1arXiv:2611.02430

InteractionBench: Measuring Timing, Task Accuracy, Whisper and Volume, and Grounding in Real-Time Voice Systems

Richard Yucheng He, Chen Xu, Yihang Liu, Tairan Chen, Baodong Cao

原文摘要

Most benchmarks for real-time voice systems focus on one aspect, such as turn-taking, instruction following, paralinguistic control, or conversational memory, each with its own data and set of systems. We present InteractionBench, a benchmark and evaluation harness that measures four dimensions on the same systems from the same recorded user audio: conversational timing, spoken-task accuracy, whisper and volume, and groundedness in conversations of more than twenty turns. The benchmark contains 221 items per system in thirteen scenarios. Before any system runs, we fix the expected floor-control actions for spoken tasks by rule and take the expected answers from scripted dialogues. We score whispering from the audio signal. We evaluate GPT-Live, GPT-Realtime, Gemini Live, Moshi, PersonaPlex, Freeze-Omni, and a turn-based cascade in 1,519 graded runs. Among the six full-duplex systems, timing and spoken-task accuracy are rank-correlated ($ρ= 0.83$), while the cascade has the highest spoken-task score (38/50) and the second-lowest timing score. No system corrects a user's grammar within the required window. Only Gemini Live whispers on most requests (8/10). Hosted systems pass 67-88% of groundedness probes, compared with 14-35% for open full-duplex models. Timing is measured where the client hears each system, so the timing of hosted systems includes network delay: for GPT-Live, whose server reports its own clock, removing a delay of about 1.5 s lowers its timing score from 86% to 56%. We also document where the protocol differs across systems, including the text instructions.

cs.SD · cs.CL · eess.AS

版本更新 · v5arXiv:2509.24773

ViSSFlow: Unifying Video-Conditioned Sound and Speech Generation via Joint Learning

Xin Cheng, Yuyue Wang, Xihua Wang, Yihan Wu, Kaisi Guan, Yijing Chen, Peng Zhang, Xiaojiang Liu 等 10 位作者

原文摘要

Video-conditioned audio generation, including Video-to-Sound (V2S) and Visual Text-to-Speech (VisualTTS), has traditionally been studied as two separate tasks despite their shared objective of generating audio conditioned on visual content. Existing methods rely on task-specific architectures and training paradigms, while the possibility of a unified generative framework remains largely unexplored due to heterogeneous conditioning signals and limited joint training data. In this paper, we propose ViSSFlow, a unified flow-matching framework for both V2S and VisualTTS. To effectively integrate diverse conditioning modalities within a Diffusion Transformer (DiT), we introduce a disentangled condition aggregation mechanism that exploits the complementary roles of attention layers, using cross-attention to model semantic conditions and self-attention to capture temporally dense visual cues. Furthermore, contrary to the common assumption that joint optimization of V2S and VisualTTS causes task interference, we show that a single model can be trained end-to-end without sacrificing task-specific performance. To address the scarcity of paired training data covering both sound and speech generation, we further propose a simple yet effective feature-level data synthesis strategy that enables robust joint learning. Extensive experiments on V2S, VisualTTS, and joint generation benchmarks demonstrate that ViSSFlow consistently outperforms state-of-the-art task-specific baselines while providing a unified solution for video-conditioned audio generation. Code and demos are provided in Supplementary Material.

eess.AS · cs.AI · cs.CL · cs.CV · cs.SD

新论文 · v1arXiv:2611.02408

Deployment-constrained key estimation with a DSP-guided transformer reranker

Pardis Noorzad

原文摘要

Local music applications estimate key during library import and benefit from an analyzer that is small, fast, and runs in-process. We start from a compact chroma-template key analyzer that fits this budget but lags behind learned references. Learned systems reach higher accuracy, but their installed model and runtime dependencies can occupy tens to hundreds of megabytes. We present key lime, a 21.5k-parameter DSP-guided transformer reranker with a 115 KB model artifact and a 76.7 MB installed footprint, including the extracted ONNX Runtime. It adds 0.32 ms per track to deterministic DSP scoring. Trained from labels on 4,040 FMAK tracks, with 449 tracks for validation and 1,000 disjoint FMAKv2 tracks held out for evaluation, it improves MIREX from 61.08 to 73.55 +/- 0.46 and exact accuracy from 52.30 to 67.07 +/- 0.65 over three seeds (mean +/- population SD). It approaches a same-manifest local madmom KeyCNN reference at 74.16 MIREX. A 12-parameter circular input stage matches the accuracy range of a 468-parameter linear input projection.

cs.SD · cs.LG · eess.AS

新论文 · v1arXiv:2611.02330

Spatial Transfusion: From Structured Spatial Scene Understanding to Generation and Editing

Jaeyeon Kim, Sebastian Braun, Hannes Gamper

原文摘要

Spatial audio conveys not only what sounds are present, but where they originate and how they interact with the environment. Recent audio-language models increasingly understand and reason about such scenes, yet spatial audio understanding, generation, and editing remain largely separate. We present Spatial Transfusion, a unified first-order ambisonics (FOA) framework that extends a pretrained spatial audio understanding model to generation and editing through flow matching over VAE latents. We formulate editing as a structured scene transformation over a taxonomy of atomic edits. Keeping the pretrained backbone frozen, we introduce modality-based LoRA routing and a curriculum that progressively adds generation, editing, and natural-language interaction. Experiments show that Spatial Transfusion largely preserves spatial understanding while outperforming full fine-tuning in generation and editing. The resulting model supports controllable FOA generation and editing through both structured and natural-language interfaces. Audio samples are available at https://microsoft.github.io/spatial-transfusion/

cs.SD · cs.AI · eess.AS

新论文 · v1arXiv:2611.02235

High Frequency Safeguard and Weighted Objective Scoring Bandwidth Extension

Tarikul Islam Tamiti, Sajid Fardin Dipto, Anomadarshi Barua

原文摘要

Speech Bandwidth Extension (BWE) reconstructs missing high frequencies from a band-limited signal. Please note that perceptually important information is often concentrated in short high-frequency (HF) events. Therefore, an effective BWE requires both accurate HF generation and explicit control over where, how often, and in what spectral form HF energy should be produced. To address the above HF supervision, we present HFSWO, a compact complex-spectral Swin-based dual-stream adversarial framework with two newly proposed specialized discriminators: 1. High-Frequency Safeguard Discriminator (HFSD) converts excessive framewise HF-energy ratios into a differentiable safeguard-constraint penalty optimized through a projected primal--dual augmented-Lagrangian update; 2. Weighted Objective Scoring Discriminator (WOSD) learns an interpretable convex objective scoring over spectral flatness, centroid, and rolloff. Our rigorous experiments on VCTK English and MLS French datasets across multiple frequency ranges, including noisy conditions, show significantly improved perceptual scores with 74.6% fewer parameters than the existing baseline AP-BWE, establishing a new baseline in the BWE task in multilingual and multispeaker setups.

cs.SD · eess.AS

版本更新 · v2arXiv:2610.38960

Efficient Agentic Audio-Visual Question Answering through Toolset Pruning and Modality Consolidation

Yi Wang, Tianzi Wang, Peter Bell

原文摘要

Fine-grained audio-visual question answering (AVQA) requires models to resolve complex cross-modal correspondences and temporally grounding from sparsely distributed evidence. Omni-modal agents, which autonomously orchestrate perception tools to actively collect evidence, achieve strong performance, but it remains unclear whether active perception from both audio and visual modalities depends on broad specialized toolsets, especially audio or visual modality-specific tools. Such systems may also introduce long execution trajectories and substantial inference cost. We investigate whether comparable performance can be retained with a compact workflow exposing only essential perceptual capabilities. Starting from the specialized-tool OmniAgent, we conduct staged controlled experiments that prune the toolset and progressively consolidate modality-specific tools into omni-modal tools, motivating a two-tool agent comprising only Omni-modal captioning and Omni-modal QA. Across three widely used AVQA benchmarks, our compact agent achieves comparable or better accuracy than the specialized-tool agent while reducing inference cost by around 70%.

cs.AI · cs.CL · eess.AS

新论文 · v1arXiv:2611.02081

Understanding the relationship between retrospective soundscape attributes and continuous pleasantness and eventfulness ratings

Yuanbo Hou, Kening Guo, Francesco Aletta, Woon-Seng Gan, Dick Botteldooren, Jian Kang

原文摘要

Soundscape studies ask listeners to rate perceived affective quality (PAQ) attributes, such as pleasant and vibrant, after listening to recordings or experiencing acoustic environments. These retrospective ratings summarise an experience during which impressions may change. International Organization for Standardization (ISO) procedures convert these PAQ ratings into Pleasantness and Eventfulness scores. Pleasantness and Eventfulness (P/E) can also be rated directly and continuously during listening, capturing changes in reported perception over time. Retrospective PAQ ratings and continuous direct P/E describe the same soundscape experience from different perspectives. We examine how PAQs relate to direct P/E and how ISO-derived P/E corresponds to direct judgements, using 1,782 paired 30-s trials involving audiovisual recordings from 18 streets. The results show that PAQs retain a predominantly two-dimensional structure. Vibrant and Monotonous lie closer to the Eventfulness-Uneventfulness axis in the direct P/E plane than in the ISO 2025 reference geometry. The full 30-s direct P/E mean has lower calibrated error against ISO-derived P/E than three 10-s means. An empirical mapping using the same 8 PAQs reduces held-out joint root-mean-square error by 4.9% relative to affine-calibrated ISO 2025 scoring. The findings show that retrospective and continuous assessments offer related but distinct soundscape descriptions.

eess.AS · cs.SD

新论文 · v1arXiv:2611.01843

MEX: The Musical Expectation Benchmark Suite

Mathias Rose Bjare, Augustin Bouquillard, Giorgia Cantisani, Seung-Goo Kim, Stefan Lattner, Gerhard Widmer

原文摘要

We propose the Musical Expectation (MEX) benchmark suite for perceptually validating computational models of musical expectation, i.e., models that estimate music information dynamics (MID) measures of perceived musical expectations, such as surprisal or uncertainty from symbolic music or musical audio. MEX collects and extends tasks for evaluating such models previously proposed in music information retrieval and cognitive sciences using standard datasets, and evaluates whether: (1) proposed MID measures decrease with repetition due to in-context learning of the computational model, (2) peaks in such measures align with boundaries of music segments, (3) pitch surprisal and uncertainty estimated from audio are consistent with those modeled in event-based symbolic music, (4) proposed MID measures can predict electrophysiological responses to music. We stress that this benchmark suite does not provide an exhaustive computational characterization of music expectation. Rather, we envision this as an active, interdisciplinary initiative, with benchmarks evolving as our understanding of the underlying perceptual processes deepens. We provide open-access code https://github.com/musicperceptmodeling/mex to estimate surprisal and uncertainty from audio using baseline models, run the tasks on standard datasets, and include utilities for running the tasks with users' own data.

cs.SD · cs.AI · eess.AS

新论文 · v1arXiv:2611.01757

Spatially-Guided Speaker Extraction Based on Relative Position Cues

Aviad Eisenberg, Shlomo E. Chazan, Sharon Gannot

原文摘要

We present a multi-channel target speaker extraction (TSE) framework that is guided by relative spatial cues rather than explicit speaker-dependent references, such as clean enrollment utterances or precise directions of arrival (DOA). The proposed approach encodes the target speaker through discrete, interpretable relations between concurrent speakers, such as left and right or near and far, referred to as the Relative Position Cue (RPC). This relational conditioning effectively leverages spatial information while avoiding explicit DOA estimation, thereby enabling more robust operation under realistic conditions. We evaluate the proposed method across multiple datasets encompassing diverse acoustic conditions, including directional and diffuse babble noise, varying numbers of microphones, mixtures with different numbers of speakers, and dynamic scenarios involving moving speakers. Experimental results demonstrate that the RPC-based approach consistently achieves strong extraction performance and generalizes well across all tested configurations, establishing it as a practical and interpretable alternative for spatially informed speaker extraction.

cs.SD · eess.AS

新论文 · v1arXiv:2611.01721

GDBack: A Group-Delay Backdoor Attack on Speaker Identification via Analytic All-Pass Filtering

Bicheng Xiong, Tao Li, Chenhui Cui, Zedong Xing, Qien Li, Weiping Wen, Rubing Huang, Tsong Yueh Chen

原文摘要

Existing phase-based backdoor attacks on speaker identification rely on shared absolute phase patterns for cross-sample trigger consistency, while other audio backdoor attacks have explored shared transformations that produce input-dependent triggers. Whether phase-based triggers can achieve similar consistency without imposing identical output phase patterns remains unclear. We propose GDBack, a poison-only group-delay backdoor attack using cascaded second-order analytic all-pass filters. Applying a shared transformation across utterances ideally preserves spectral magnitude while introducing a consistent frequency-dependent phase response and group-delay contribution, without forcing identical output phases. Experiments on LibriSpeech and VoxCeleb1 with ECAPA-TDNN, x-vector, and RawNet3 achieve attack success rates above 94% across all six settings (97.00% on average), while maintaining benign accuracy close to clean models. Phase and magnitude interventions, inverse compensation, and parameter-sensitivity analyses support the role of the induced phase transformation in backdoor activation. Speech quality and content evaluations indicate limited perceptual and linguistic distortion, while robustness experiments, including real over-the-air tests, reveal varying effectiveness across conditions and architectures. These findings demonstrate that cross-sample trigger consistency can arise from a shared phase transformation without requiring a common absolute phase pattern.

cs.SD · eess.AS

版本更新 · v2arXiv:2610.09737

Mosquito acoustic identification at unseen recording sites: practical limits of common transfer remedies for vector surveillance

Yucheng Gong, Rui Zhou, Binbin Zeng, Qiang Ren, Hongjin Hui

原文摘要

Passive acoustic monitoring could widen mosquito surveillance, but a classifier trained under one set of recording conditions often fails when moved to a new site, device or enclosure. On a nine-species, five-condition corpus of about 271,000 recordings, the published baseline identifies species at 0.881 balanced accuracy under conditions it has seen and 0.175 under held-out ones. Using that corpus's leave-domain protocol, we asked which transfer strategy is most reliable when a program brings a new site online. Using fixed features from a pre-trained bioacoustic model and a small classifier improved held-out-condition balanced accuracy to 0.305, without updating backbone weights and in minutes of training per site. Four widely used additions did not improve transfer: rebalancing training across recording conditions, fine-tuning the backbone and fusing a weaker second modality each reduced accuracy, and a six-term objective reduced to a single alignment term. Rebalancing reversed direction across four encoders, helping weak representations and harming strong ones. Genus-level aggregation was more accurate than species-level classification (about 0.50 against 0.30), suggesting genus-level screening may be the more realistic operating point here. These results come from one corpus under one protocol, and are dataset-bounded guidance rather than validated general rules.

eess.AS · cs.LG

新论文 · v1arXiv:2611.01639

Local or Escalate? Post-Training Expert Routing for Real-Time Duplex Speech Models

Ruoyu He, Jisen Li, Anthony Ji, Changyi Yang, Zetian Li, Runduo Ma, Zhen Zhang, Zhongzhu Zhou

原文摘要

A full-duplex model should answer easy questions immediately, but hard questions often need stronger reasoning, which may require waiting. The challenge is to choose between these paths at the moment the model begins to speak. Recent systems learn related mode decisions through special-token generation inside the dialogue model, coupling routing to turn-taking and temporally annotated full-duplex training. We study a post-training alternative. We freeze MiniCPM-o 4.5, using only its speech input and output, and fit a logistic failure readout on hidden states already available at its native listen-to-speak transition. The resulting gate requires no additional model generation and exposes thresholds that can be retuned without changing the duplex policy. A probe fitted on text-input states also reads the states of spoken questions, including unseen human recordings. On the internal benchmark, selective handoff improves answer accuracy by 27.1 points. Compared with always using the expert, it reduces mean server time to first answer audio by 31.5 percent. It also outperforms matched random routing across external speech question-answering pools, as well as a zero-shot GPT-5.4 mini router that rates the transcribed question. The training recipe also transfers to a second duplex model family. These results show that an existing duplex model can be equipped to answer easy turns immediately and wait for stronger reasoning or fresher knowledge on the turns that need it, without further duplex training.

cs.AI · eess.AS

版本更新 · v3arXiv:2603.08683

Benchmarking Language Modeling for Lossless Compression of Full-Fidelity Audio

Phillip Long, Zachary Novack, Chris Donahue

原文摘要

Autoregressive "language" models (LMs) trained on raw waveforms can be repurposed for lossless audio compression, but prior work is limited to 8-bit audio, leaving open whether such approaches work for practical settings (16/24-bit) and can compete with existing codecs. We benchmark LM-based compression on full-fidelity audio across diverse domains (music, speech, bioacoustics), sampling rates (16kHz-48kHz), and bit depths (8, 16, 24-bit). Standard sample-level tokenization becomes intractable at higher bit depths due to vocabulary size (65K for 16-bit; 16.7M for 24-bit). We propose Trilobyte, a byte-level tokenization schema for full resolution audio, improving vocabulary scaling from $O(2^{b})$ to $O(1)$ and enabling the first tractable 24-bit LM-based lossless compression. While LMs consistently outperform FLAC and yield state-of-the-art compression at 8-bit and 16-bit, we observe that compression gains become more modest as bit depth increases beyond 8-bit.

cs.SD · cs.AI · cs.LG · eess.AS

新论文 · v1arXiv:2610.40505

在自回归语音克隆TTS模型中发现可解释的声学特征

Discovering Interpretable Acoustic Features in Autoregressive Voice-Cloning TTS Models

Anuj Diwan, David Harwath, Eunsol Choi

AI 速读 基于摘要

该论文提出一个基于特征的框架,用于解释自回归语音克隆TTS模型内部表征的声学属性,并评估稀疏自编码器、PCA和随机方向在探测与操控上的效果。

eess.AS · cs.CL · cs.LG · cs.SD

查看原文摘要

Voice-cloning TTS models synthesize an unseen speaker's voice from brief speech prompts, but it remains unclear which acoustic properties they represent and use. We introduce a feature-based interpretability framework for autoregressive voice-cloning TTS models, analyzing features from sparse autoencoders (SAEs), PCA, and random directions via sparse probing, steering, and quantitative auto-interpretability. We focus on Qwen3-TTS and extend probing and steering to $5$ additional models to test generalization across architectures. These models encode concepts like pitch, energy, noise, and accent, but sparse prediction and steering are effective only for some concepts. Strong probing does not always translate to steering or vice-versa. SAEs are useful for sparse probing of human-labelled concepts and novel-concept discovery, while PCA is competitive for dense probing and steering. Auto-interpretability finds $684$ automatically validated SAE features in Qwen3-TTS covering speaker clusters, speaking style, and recording conditions beyond known concepts. Overall, we perform a comprehensive interpretability study of autoregressive voice-cloning TTS models.

新论文 · v1arXiv:2610.39898

Ambisonics: Introducing an Accessible, Distributed Approach

Thomas A Rushton, Romain Michon, Tanguy Risset, Clemens Wegener, Pierre Lecomte

原文摘要

This paper presents an accessible, distributed approach to higher order ambisonics using a network of low-cost microcontroller-based devices. Leveraging recent advances in embedded computing, networked audio, and precise time synchronization, the system offers an alternative to conventional centralized architectures that typically require expensive multichannel interfaces and high-performance digital signal processing hardware. The proposed architecture combines an audio plugin for soundfield design with a modular network of ambisonics processing units, which, exploiting the parallelisable nature of ambisonics decoding, are each responsible for a subset of loudspeaker outputs. Synchronization is achieved through the IEEE 1588 Precision Time Protocol, complemented by adaptive clock correction to maintain sub-microsecond alignment between devices. A third-order ambisonics implementation demonstrates the viability of the approach, with potential scalability to higher orders and larger arrays. Overall, this work contributes to the development of frugal, distributed solutions for immersive audio, expanding access to ambisonics-based systems for a wide range of applications.

cs.SD · eess.AS

新论文 · v1arXiv:2610.39754

Toward Complete Speaker Anonymization: a Streaming Cascaded System for Voice Privacy and PII Removal

Stanisław Kacprzak, Thomas Thebaud, Najim Dehak

原文摘要

We present a cascade streaming voice privacy system, consisting of ASR, NLP, and TTS modules. By leveraging recent improvements in all three aspects of speech and language processing, we show that such a system is capable of achieving a result comparable to state-of-the-art offline voice privacy systems, while providing controllable capabilities of content protection beyond voice. With an estimated utterance-onset end-to-end latency of 717 ms, the presented system preserves speech content with 11% WER on Librispeech, while detecting PII with a 69% SLU F1 score and achieving up to 47.66% EER against a semi-informed attacker.

cs.SD · eess.AS

新论文 · v1arXiv:2610.39742

The Importance of Speech Genre for End-of-Turn Detector Training

Rini Sharon, Kadri Hacioglu, Jeena Prakash, Andreas Stolcke

原文摘要

End-of-turn (EoT) detection has been studied mostly through features, models, and modalities, while the speech genre of the training data has drawn far less attention. We isolate this factor. Using one reproducible annotation pipeline, we build a matched pair of corpora, multi-turn human-human conversational and single-turn human-machine, and train the same streaming ConvLSTM EoT detector on each. Across four test sets the conversational model reaches up to +0.45 higher F1, driven by fewer false alarms on the mid-turn pauses and hesitations that are rare in single-turn corpora. This conversational-training effect is not specific to frame-level decoding. Under a matched complete-or-not test it also appears at the utterance level, across two encoders, but only on conversational test data. Testing only on single-turn speech, as most prior work does, therefore hides the benefit of conversational training data. Against open-source baselines, our 1.8M-parameter model attains the highest F1 on every test set.

eess.AS · cs.AI

版本更新 · v2arXiv:2411.14493

From Statistical Methods to Pre-Trained Models; A Survey on Automatic Speech Recognition for Resource Scarce Urdu Language

Muhammad Sharif, Zeeshan Abbas, Jiangyan Yi, Chenglin Liu

原文摘要

Automatic Speech Recognition (ASR) technology has witnessed significant advancements in recent years, revolutionizing human-computer interactions. While major languages have benefited from these developments, lesser-resourced languages like Urdu face unique challenges. This paper provides an extensive exploration of the dynamic landscape of ASR research, focusing particularly on the resource-constrained Urdu language, which is widely spoken across South Asian nations. It outlines current research trends, technological advancements, and potential directions for future studies in Urdu ASR, aiming to pave the way for forthcoming researchers interested in this domain. By leveraging contemporary technologies, analyzing existing datasets, and evaluating effective algorithms and tools, the paper seeks to shed light on the unique challenges and opportunities associated with Urdu language processing and its integration into the broader field of speech research.

cs.CL · cs.SD · eess.AS

新论文 · v1arXiv:2610.39427

EmpDPO: Enabling Emphasis Control in TTS via Direct Preference Optimization

Haonan Chen, Yunyun Wang, Megan Wei, Zeyu Jin

原文摘要

Word-level emphasis conveys meaning that text alone cannot express and is therefore essential for a practical text-to-speech (TTS) system. In this work, we study how to add emphasis control post hoc to a pretrained TTS model that has never been trained with emphasis annotations. We present EmpDPO, a simple, efficient, and effective two-stage recipe that adds emphasis control to a multilingual zero-shot diffusion TTS model through supervised fine-tuning followed by direct preference optimization (DPO). The training signals for both stages come from an off-the-shelf multilingual emphasis detector, so that no manual emphasis annotation or task-specific reward design is needed. We find that DPO can substantially strengthen emphasis control with only a few thousand preference pairs and training steps. Objective and subjective evaluations against strong public systems show that our model produces perceptible emphasis while preserving naturalness and overall quality.

cs.SD · eess.AS

新论文 · v1arXiv:2610.39168

Reading-Level Verification of Kanji Words in Japanese Financial Speech Synthesis

Cheng Zhang

原文摘要

Few-shot voice cloning makes it cheap to produce daily Japanese financial commentary with several synthetic speakers, but kanji words with more than one reading, such as the word for "market" (shijo vs. ichiba) or "overall trend" (taisei vs. oze), are still misread. A common quality gate transcribes each synthesized sentence with automatic speech recognition (ASR), converts both the input and the transcript to kana, and regenerates on mismatch. We show that this gate is unreliable in both directions: when the audio contains a misreading, the ASR language model frequently writes the expected kanji back, hiding the error, and when the audio is correct, ordinary recognition errors trigger needless regeneration. We propose a reading-level check that scores the expected reading and its confusable alternatives with a connectionist temporal classification (CTC) hiragana recognizer and flags a word when an alternative explains the audio better. On 540 utterances from three voice-cloned production voices covering 30 financial terms, with misreadings injected under controlled conditions, the transcript-based gate detected 78.9% of misreadings at a 44.4% false-alarm rate, whereas the proposed check detected 91.7% at 5.0% (AUC 0.990). On natural kanji input, it caught a front-end error that the transcript-based gate missed for one voice, and reduced regeneration requests from 38.9% to 6.1% of sentences.

cs.SD · cs.CL · eess.AS

新论文 · v1arXiv:2610.38739

Streaming Speech Recognition on a Microcontroller: a 65.7M-Parameter Transducer in 2 and 4 Bits on the ESP32-P4

Billal Boumedine

原文摘要

We run Kroko, a 65.7M-parameter streaming Zipformer2 transducer for English, on the ESP32-P4, a microcontroller with two 400 MHz RISC-V cores, a 128-bit SIMD extension (PIE) and 32 MB of PSRAM, but no neural accelerator. Without any training, the encoder's 290 matrices are stored with 2 or 4 bits per weight in Hadamard-rotated groups: the width of each matrix comes from its measured sensitivity, and each matrix is first corrected toward the original model's output, then rounded with GPTQ and a search over the group norms. The released 71 MB of int8 weights become a 26.65 MB file. Activations are int16 between the matrix products and int8 into them. A C engine with hand-written PIE kernels processes each 1.28 s chunk of audio in 0.90 s on both cores, beam search included, and its PC build computes the same bits as the chip, so accuracy is measured on whole test sets. LibriSpeech WER is 4.34% on test-clean and 10.26% on test-other, against 3.94% and 9.17% for the released model; on FLEURS and Common Voice the gap grows to 2.3 and 1.1 points. A GPU simulator shows that the integer arithmetic costs nothing and that the whole gap comes from the weights, and an ablation on all of test-clean shows that GPTQ does most of the work. We also report the port's speed history, from 51.3 s to 0.62 s of encoder time per chunk, each design step measured again in one build, an attempt to reach 16 MB that stopped at 19.63 MB, and what did not work. We did not measure energy. The inference engine, the firmware and the 2/4-bit weights are available at https://github.com/BoumedineBillal/esp32-pie-asr.

eess.AS · cs.AR · cs.LG