← 研踪 PaperTrailarXiv:2506.03013 · v3
Cataloguing Hugging Face Models to Software Engineering Activities: Automation and Findings
首次提交 2025-06-03 23:51
版本更新 2026-11-10 02:29
雷达观测 2026-11-10 15:22
摘要
Context: Open-source Pre-Trained Models (PTMs) provide extensive resources for various Machine Learning (ML) tasks, yet these resources lack a classification tailored to Software Engineering (SE). Objective: To address this gap, we derive a taxonomy encompassing 146 SE tasks and apply an SE-oriented classification to PTMs in a popular open-source ML repository, Hugging Face (HF). Method: Our repository mining study followed a five-phase pipeline: (i) identification of SE tasks from the literature; (ii) collection of PTM data from the HF API, including model card descriptions and metadata, and the abstracts of the associated arXiv papers; (iii) text processing to ensure consistency; (iv) a two-phase validation of SE relevance, involving humans and LLM assistance, supported by five pilot studies with human annotators and a generalization test; and (v) data analysis. This process yielded a curated catalogue of 5,027 SE PTMs. Results: We find that most SE PTMs target code generation and coding, emphasizing implementation over early or late development stages. In terms of ML tasks, text generation dominates within SE PTMs. Notably, the number of SE PTMs has increased markedly since 2023 Q2, while benchmarking remains limited: only 5.55\% of all SE PTMs in our catalogue provide evaluation results. Conclusions: Our catalogue reveals documentation and transparency gaps, highlights SE activity imbalances, and supports automated SE scenarios, such as the selection of suitable PTMs.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v3 · 2026-11-10 02:29 · 最新观测版本
Cataloguing Hugging Face Models to Software Engineering Activities: Automation and Findings
Context: Open-source Pre-Trained Models (PTMs) provide extensive resources for various Machine Learning (ML) tasks, yet these resources lack a classification tailored to Software Engineering (SE). Objective: To address this gap, we derive a taxonomy encompassing 146 SE tasks and apply an SE-oriented classification to PTMs in a popular open-source ML repository, Hugging Face (HF). Method: Our repository mining study followed a five-phase pipeline: (i) identification of SE tasks from the literature; (ii) collection of PTM data from the HF API, including model card descriptions and metadata, and the abstracts of the associated arXiv papers; (iii) text processing to ensure consistency; (iv) a two-phase validation of SE relevance, involving humans and LLM assistance, supported by five pilot studies with human annotators and a generalization test; and (v) data analysis. This process yielded a curated catalogue of 5,027 SE PTMs. Results: We find that most SE PTMs target code generation and coding, emphasizing implementation over early or late development stages. In terms of ML tasks, text generation dominates within SE PTMs. Notably, the number of SE PTMs has increased markedly since 2023 Q2, while benchmarking remains limited: only 5.55\% of all SE PTMs in our catalogue provide evaluation results. Conclusions: Our catalogue reveals documentation and transparency gaps, highlights SE activity imbalances, and supports automated SE scenarios, such as the selection of suitable PTMs.
查看此版本解读 →查看此版本原文 ↗