SPOT Research Group
About
We are a research group at the University of Michigan, primarily in the School of Information and Computer Science. Our research focuses on understanding the factors that make programming tools usable and designing & building new tools for programmers and people learning to program.
What we work on: Programming Education at Scale Web Automation and Programming by Demonstration Building and Testing Interactive Interfaces Explaining Code and Capturing Process Accessibility of Programming and the Web Immersive Authoring On-Demand Help for Programmers
News
-
Aug '26:
Steve Oney is beginning his term as MSI Program Committee Chair at the University of Michigan School of Information.
-
Aug '26:
Steve Oney just launched a new Coursera specialization: Practical JavaScript: From Scratch to React
-
Jul '26:
Dr. Maryam Arab will join SUNY Albany as a Visiting Assistant Professor in September. Congratulations, Professor Arab!
-
Jul '26:
Ashley’s paper on visualizing student-AI interaction in code was accepted to UIST 2026!
-
Ashley Zhang
- Yan-Ru Jhou
- Yinuo Yang
- Shamita Rao
-
Maryam Arab
-
Yan Chen
-
Editrail: Understanding AI Usage by Visualizing Student-AI In...
-
People
Recent Publications
-
Editrail: Understanding AI Usage by Visualizing Student-AI Interaction in Code
Editrail visualizes AI contributions alongside students’ code editing histories to help instructors understand how students use and adapt generated code. In a comparison study, participants identified AI usage patterns more accurately than with the baseline interface and used the histories to decide where targeted guidance could help.
-
CodeStream: Augmenting Timelines with Code Annotation for Navigating Large Coding Histories
CodeStream combines code annotations, semantic zooming, and cumulative effort indicators to help instructors navigate students’ programming histories. In a study with 12 participants, it improved accuracy in identifying coding patterns compared with a baseline timeline while requiring comparable analysis time.
-
Navigating Complexity: How Context Shapes Debugging Strategy Choices Among Expert Developers
A survey of 35 web developers and interviews with 16 expert web developers examine how context shapes debugging strategy choices. The study identifies static and dynamic contextual factors and a descriptive state-transition model showing how experts adapt their strategies as clarity, reproducibility, and constraints change.
-
Co-Advisor: Learning Programming Strategies in Context
Co-Advisor uses code context, runtime information, and programmer actions to provide debugging strategies with personalized feedback in Visual Studio Code. In a formative study with 14 programmers, participants made more debugging progress with contextual feedback than with non-adaptive strategies, suggesting a way to support strategic knowledge during problem solving.
-
ConvoMap: Interactive Visualizations for Exploring Complex Conversations in Multi-Agent Systems
ConvoMap helps developers inspect conversations among AI agents through automated topic coding, a semantic map, and linked message views. In a comparison study with 16 participants, it improved accuracy in understanding agent behavior and conversation patterns, supporting investigation of recurring problems across multiple runs.
-
SPARK: Real-Time Monitoring of Multi-Faceted Programming Exercises
SPARK helps instructors monitor programming exercises with interdependent steps and varied outputs by organizing progress around instructor-defined checkpoints. It suggests tests, visualizes progress, and exposes intermediate outputs; a study with 16 participants found more accurate identification of student difficulties than with the baseline dashboard.
-
Multi-Click: Cross-Tab Web Automation via Action Generalization
Multi-Click lets users select analogous elements within or across browser tabs and apply clicks or text entry to them together. Structural and visual matching, target previews, and an editable data table help users control repeated web actions and vary inputs without writing an automation script.
-
VRCopilot: Authoring 3D Layouts with Generative Models in VR
VRCopilot combines generative AI, speech and pointing, and editable wireframes to help users create room layouts in virtual reality. Its studies found that wireframe-based creation supported greater perceived agency than automatic generation, while manual creation through multimodal specification offered the strongest sense of creativity and control.
-
ScrapeViz: Hierarchical Representations for Web Scraping Macros
ScrapeViz lets users create web scraping macros by demonstration and inspect their behavior through a hierarchical visual storyboard. In a study with 12 participants, its linked page views and output table helped users trace scraped data, identify anomalies, and check macros while authoring.
-
CFlow: Supporting Semantic Flow Analysis of Students' Code in Programming Problems at Scale
CFlow helps instructors compare large collections of student programs through groups of semantically related statements, aligned code views, and visual summaries of common mistakes. In a study with 16 participants analyzing more than 6,000 submissions, participants answered questions about mistakes and patterns faster and identified more valid mistakes than with the baseline system.
-
Towards Inclusive Source Code Readability Based on the Preferences of Programmers with Visual Impairments
A qualitative study with 16 blind and visually impaired developers examines preferences for code formatting, including naming conventions, line length, and indentation. The findings show where screen reader use changes readability needs and suggest improvements to programming languages, development environments, and style guides.
-
RunEx: Augmenting Regular-Expression Code Search with Runtime Values
RunEx helps programming instructors search collections of student code by combining regular expressions with constraints on values produced during execution. In a study with experienced Python programmers, most with teaching experience, participants identified coding patterns and mistakes more accurately than with tools limited to text matching.
-
Colaroid: A Literate Programming Approach for Authoring Explorable Multi-Stage Tutorials
Colaroid is a Visual Studio Code extension for authoring programming tutorials that connect explanations, code differences, and runnable versions of a project. Studies found that authors could readily create tutorials and that readers used intermediate versions to explore the code at their own pace.
-
VizProg: Identifying Misunderstandings by Visualizing Students' Coding Progress
VizProg visualizes students’ changing code on a two-dimensional map to help instructors follow progress and recognize different approaches during programming exercises. In a comparison using replays of students’ work, participants identified more misunderstandings and answered questions about students’ problems more accurately than with an adapted OverCode baseline.
-
VRGit: A Version Control System for Collaborative Content Creation in Virtual Reality
VRGit brings version control into virtual reality through a graph of miniature 3D scenes that users can navigate, branch, and reuse. A laboratory study found that its previews, shared history views, and portals helped participants manage design alternatives and discuss changes while collaborating on virtual room layouts.
-
Accessibility of UI Frameworks and Libraries for Programmers with Visual Impairments
This paper examines the accessibility of UI frameworks and libraries through a mailing-list analysis and interviews with 18 programmers with visual impairments. It shows how interactions among frameworks, development tools, operating systems, and screen readers complicate coding and collaboration, and recommends clearer accessibility documentation and better support for seeking help.
-
ParamMacros: Creating UI Automation Leveraging End-User Natural Language Parameterization
ParamMacros lets people create website automation by marking parameters and possible values in a natural language question, then demonstrating how to answer it. The system uses that input to generalize the demonstration, and a laboratory study found that participants could specify meaningful parameters and saw value in the approach.
-
PuzzleMe: Leveraging Peer Assessment for In-Class Programming Exercises
PuzzleMe supports peer assessment during programming exercises by letting students share lightweight tests and review code in groups matched by their solutions. Classroom deployments showed how these features helped students identify bugs, correct misunderstandings, and explore alternative approaches, while also revealing that peer discussions sometimes needed encouragement.
-
Understanding Accessibility and Collaboration in Programming for People with Visual Impairments
This interview study examines the workplace experiences of 22 programmers with visual impairments, including how they set up tools and collaborate with sighted colleagues. It shows that inaccessible software and visually oriented team practices create additional work, from developing technical workarounds to explaining access needs and negotiating changes to shared workflows.
-
Understanding the Challenges and Needs of Programmers Writing Web Automation Scripts
This paper investigates how programmers write automation scripts for third-party websites through two studies, including an evaluation of a prototype editor that shows page context and execution feedback. It identifies difficulties selecting elements consistently across pages and recommends feedback about selected elements, interaction results, and changes to websites over time.
-
CoCapture: Effectively Communicating UI Behaviors on Existing Websites by Demonstrating and Remixing
CoCapture lets designers communicate changes to an existing website by recording its behavior, demonstrating and editing new behavior, and linking written requests to the resulting mockups. Study participants described desired UI behavior more accurately than with conventional sketching and communication tools, and reviewers found those descriptions easier to follow.
-
Think-Aloud Computing: Supporting Rich and Low-Effort Knowledge Capture
Think-aloud computing captures spoken explanations, design decisions, and unresolved problems while people work, using prompts and links to application context. Studies across programming, slide creation, and 3D modeling found that it captured information missing from traditional documentation, with participants reporting similar overall effort but different sources of disruption.
-
FlowMatic: An Immersive Authoring Tool for Creating Interactive Scenes in Virtual Reality
FlowMatic lets programmers create interactive virtual reality scenes while immersed in them, using visual dataflow connections to define reactions to events, create objects, and reuse behaviors. A study with eight programmers found that participants could build the requested scenes and valued the direct manipulation and immediate feedback.
-
A Hybrid Crowd-Machine Workflow for Program Synthesis
BashOn combines automated shell-command synthesis with crowd workers who identify parts of natural language requests and check the outputs of candidate commands. Experiments showed that these targeted contributions improved command-generation accuracy over the automated system alone, even though the workers lacked Bash expertise, while leaving some requests without a returned answer.
-
EdCode: Towards Personalized Support at Scale for Remote Assistance in CS Education
EdCode lets programming students request help within their code editor and lets instructors link explanations to relevant code, follow up through chat, and share selected answers with the class. Usability studies found that students valued the contextual support and could understand shared answers even when unrelated code was hidden.
-
Sifter: A Hybrid Workflow for Theme-based Video Curation at Scale
Sifter combines automated video filtering with crowd workers who select videos and reach agreement on material for themed compilations. A comparison covering 12 compilations found faster video selection than staff curators and similar relevance ratings for 11 compilations, showing how the workflow can assist editorial curation.
-
Callisto: Capturing the "Why" by Connecting Conversations with Computational Narratives
Callisto connects discussions to notebook cells and their editing history so collaborators can recover the reasoning behind a data analysis. In a study with data science students, these links helped newcomers understand an ongoing project more accurately, including earlier alternatives that no longer appeared in the final notebook.
-
Explore, Create, Annotate: Designing Digital Drawing Tools with Visually Impaired People
This study examines how 11 adults with visual impairments explore, create, and label tactile maps using braille and audio. Audio labels supported faster answers to questions about the maps, while braille supported more varied exploration strategies; the findings inform drawing tools that accommodate both labeling methods and familiar interactions.
-
Improving Crowd-Supported GUI Testing with Structural Guidance
This paper presents interactive event-flow graphs and guidance within the interface to help crowd testers find unexplored interaction sequences. In a study with 30 testers across 11 test pages, the techniques increased transition coverage and prevented repeated transitions, helping both trained and untrained testers make more thorough tests.