Nicole Sullivan, Ella Cao, Leilani Battle, Magdalena Balazinska
原文摘要
While LLMs are increasingly used in various contexts, we lack a fundamental understanding of how they impact SQL query debugging. In response, we conducted a user study with 16 participants of varying SQL expertise, examining how they debug complex SQL queries with and without LLM support. Initial benchmarking indicated that our selected models are already capable SQL debuggers. Qualitative analyses of our study data revealed that while participants trusted the LLM, they varied widely in how and whether they delegated debugging tasks to the LLMs, which we characterized through a new three-state model of participants' SQL reasoning processes. Additionally, participants enforced specific collaborative roles for the LLM such as TA or sounding board, which participants controlled by withholding context from the LLM. These findings suggest that LLM-based tools could enable preset role selection, broader context management, and flexible automation features, including low automation options, to support SQL debugging moving forward.
Dashboard creation requires iteratively authoring multiple views and organizing them into a structured format. However, creators often struggle to step back and assess their own work for organization and refinement due to cognitive limitations. In this work, we propose a pipeline to provide computational support for self-reflection during the design process. Our pipeline automatically infers analytical objectives, discovers implicit patterns, and structures those patterns to surface coherent analytical pathways. It further enriches existing dashboard designs by generating complementary visualizations and aggregating natural language insights based on relationships across data entities (e.g., contrasting, drill-down, and relatedness) to reveal chart connections. Our rule-based pipeline provides objective, verifiable feedback, offering an external perspective while enabling creators to tell their own data stories. Four case studies and six expert interviews demonstrate how this approach helps creators reflect on organizational alternatives and surface hidden analytical opportunities in a principled way, reducing the cognitive burden while encouraging deeper analytical exploration.