← 研踪 PaperTrailarXiv:2611.14798 · v1
Who (or what) are they prompting? How CS students conceptualize and use LLMs
首次提交 2026-11-11 06:34
版本更新 2026-11-11 06:34
雷达观测 2026-11-13 10:09
摘要
Large language models (LLMs) are increasingly used by students in computing courses. Yet computing education research has not fully examined how teaching and inclusion of LLMs shape students' conceptualization and understanding of LLMs, and how these factors relate to their subsequent use. Although prior work has examined prompting strategies and AI literacy, the relationships among students' conceptualizations of LLMs, their understanding of LLM mechanisms, and their prompting behavior remain underexplored. We studied these relationships in a post CS1 course that introduces students to the various CS subfields. We adapted the course to teach and allow the usage of LLMs. We administered an end-of-course survey, using open-ended items to elicit students' representations of LLMs and their explanations of how LLMs work, alongside closed-response measures of prompting style and prompt count (N=283 students consented and provided codable responses). Using qualitative coding, we categorized students' understanding of how LLMs work into four ordinal knowledge levels based on the Structure of Observed Learning Outcomes (SOLO) taxonomy and coded their metaphoric conceptualizations along two dimensions: persona type and relational role. Our results show that students most often conceptualized LLMs as human assistants, and context-rich prompting was the preferred style. The study contributes a two-dimensional characterization of students' LLM metaphoric representation and exploratory analyses also identified associations among these constructs showing how understanding these systems relates to their usage.
作者备注:Accepted to SIGCSE 2027
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-11 06:34 · 最新观测版本
Who (or what) are they prompting? How CS students conceptualize and use LLMs
Large language models (LLMs) are increasingly used by students in computing courses. Yet computing education research has not fully examined how teaching and inclusion of LLMs shape students' conceptualization and understanding of LLMs, and how these factors relate to their subsequent use. Although prior work has examined prompting strategies and AI literacy, the relationships among students' conceptualizations of LLMs, their understanding of LLM mechanisms, and their prompting behavior remain underexplored. We studied these relationships in a post CS1 course that introduces students to the various CS subfields. We adapted the course to teach and allow the usage of LLMs. We administered an end-of-course survey, using open-ended items to elicit students' representations of LLMs and their explanations of how LLMs work, alongside closed-response measures of prompting style and prompt count (N=283 students consented and provided codable responses). Using qualitative coding, we categorized students' understanding of how LLMs work into four ordinal knowledge levels based on the Structure of Observed Learning Outcomes (SOLO) taxonomy and coded their metaphoric conceptualizations along two dimensions: persona type and relational role. Our results show that students most often conceptualized LLMs as human assistants, and context-rich prompting was the preferred style. The study contributes a two-dimensional characterization of students' LLM metaphoric representation and exploratory analyses also identified associations among these constructs showing how understanding these systems relates to their usage.
查看此版本解读 →查看此版本原文 ↗