SPOT Research Group
About
We are a research group at the University of Michigan, primarily in the School of Information and Computer Science. Our research focuses on understanding the factors that make programming tools usable and designing & building new tools for programmers and people learning to program.
What we work on: Programming Education at Scale Web Automation and Programming by Demonstration Building and Testing Interactive Interfaces Explaining Code and Capturing Process Accessibility of Programming and the Web Immersive Authoring On-Demand Help for Programmers
News
-
Aug '26:
Steve Oney just launched a new Coursera specialization: Practical JavaScript: From Scratch to React
-
Jul '26:
Dr. Maryam Arab will join SUNY Albany as a Visiting Assistant Professor in September. Congratulations, Professor Arab!
-
Jul '26:
Ashley’s paper on visualizing student-AI interaction in code was accepted to UIST 2026!
-
Ashley Zhang
- Yan-Ru Jhou
- Yinuo Yang
- Shamita Rao
-
Maryam Arab
-
Yan Chen
-
Editrail: Understanding AI Usage by Visualizing Student-AI In...
-
-
Apr '26:
Several group members will be in Barcelona for CHI 2026! Ashley will present her paper on navigating coding histories at scale and Xiangyu will present his poster on simulating cursor trajectories for GUI evaluation.
People
Recent Publications
-
Editrail: Understanding AI Usage by Visualizing Student-AI Interaction in Code
Programming instructors have diverse philosophies about integrating generative AI into their classes. Some encourage students to use AI, while others restrict or forbid it. Regardless of their approach, all instructors benefit from understanding how their students actually use AI while writing code. Such insight hel...
-
CodeStream: Augmenting Timelines with Code Annotation for Navigating Large Coding Histories
Introduces a visualization system that helps instructors interpret students’ keystroke-level coding histories. While traditional timelines show when and where edits occurred, they do not reveal what changed. CodeStream augments timelines with zoomable code annotations, multi-level structural summaries, and a cumulative effort heatmap, allowing instructors to see both the content and intensity of edits without constant context switching.
-
Co-Advisor: Learning Programming Strategies in Context
A context-aware learning tool that helps programmers develop strategic problem-solving skills like debugging and program design. Integrated with Visual Studio Code, it analyzes the learner's code, runtime state, and actions to recommend expert strategies tailored to a developer's context. As learners follow these strategies, Co-Advisor provides real-time feedback to make it easier for developers to understand strategic knowledge in context.
-
ConvoMap: Interactive Visualizations for Exploring Complex Conversations in Multi-Agent Systems
A visualization system that helps developers understand and debug large collections of conversations between AI agents in Multi-Agent Systems (MASs). ConvoMap contains visualizations to show the flow of interactions between agents and the topics covered. ConvoMap allows MAS developers to spot behavioral patterns, trace errors, and compare outcomes at scale.
-
SPARK: Real-Time Monitoring of Multi-Faceted Programming Exercises
A dashboard for programming instructors that lets them monitor student progress on complex, multi-step programming exercises. SPARK lets instructors define checkpoints, automatically generates tests for each step, and visualizes progress across students. SPARK also enables inspection of intermediate outputs and runtime states to reveal common challenges and misconceptions.
-
Multi-Click: Cross-Tab Web Automation via Action Generalization
Proposes "action generalization"---a technique for applying the same action across multiple contexts automatically. Multi-click lets users select a "generalization strategy" to determine which elements a given action (e.g., click or text entry) will apply to within a given page or across multiple tabs. Unlike prior web automation tools, multi-click works at the *interaction* level, rather than requiring a separate script representation of automation macros, which can improve the directness and understandability of bulk actions.
-
VRCopilot: Authoring 3D Layouts with Generative Models in VR
A mixed-initiative tool for immersive 3D layout design in VR. It integrates generative AI to assist users in co-creating room layouts using voice commands, multimodal interactions, and wireframe-based representations. VRCopilot supports three modes: manual creation, automatic AI generation, and scaffolded creation with wireframes, balancing user control and automation.
-
ScrapeViz: Hierarchical Representations for Web Scraping Macros
A programming-by-demonstration tool that helps users create and visualize web scraping macros across multiple pages. ScrapeViz offers a visual storyboard of scraping behaviors, making it easier to understand, trace data sources, and identify errors.
-
CFlow: Supporting Semantic Flow Analysis of Students' Code in Programming Problems at Scale
Introduces CFlow, a system for visualizing and analyzing large numbers of student code submissions to help instructors understand common mistakes and patterns. CFlow uses semantic labeling and code structure analysis to cluster code lines across submissions and present them in a visualization that resembles a single code sample, allowing instructors to see the overall semantic flow while still being able to dive into individual submissions.
-
Towards Inclusive Source Code Readability Based on the Preferences of Programmers with Visual Impairments
Explores the code readability preferences of blind and visually impaired (BVI) developers and how they differ from sighted developers, through a qualitative study. Contributes an inclusive taxonomy for code readability and recommendations for programming tools, code standards, and languages to better support BVI developers.
-
RunEx: Augmenting Regular-Expression Code Search with Runtime Values
A system that improves code search by augmenting regular expressions with the ability to specify constraints on runtime values of specific expressions.
-
Colaroid: A Literate Programming Approach for Authoring Explorable Multi-Stage Tutorials
Colaroid is an extension to the Visual Studio Code IDE that makes it easier to create mutli-step tutorials about a large code sample. Developers can write their code in the IDE and as they are writing their code, Colaroid allows them to divide their work into steps. Colaroid tutorials are augmented computational notebooks, where snippets and outputs represent a snapshot of a project, with source code differences highlighted, complete source code context for each snippet, and the ability to load and tinker with any stage of the project in a linked IDE.
-
VizProg: Identifying Misunderstandings by Visualizing Students' Coding Progress
VizProg is a tool that allows instructors of large programming classes to monitor students' progress at scale. It represents students' code in a 2D map where dots representing students move further to the right as they get closer to a correct solution. VizProg uses vertical space to represent different approaches to solving the problem (so students towards the bottom of the map are using a different approach than students at the top of the map).
-
VRGit: A Version Control System for Collaborative Content Creation in Virtual Reality
A version control system designed especially for Virtual Reality (VR). VRGit allows users to view and navigate the version history of a 3-D scene while immersed in VR. It also allows users to share and discuss history graphs collaboratively.
-
Accessibility of UI Frameworks and Libraries for Programmers with Visual Impairments
Two studies into the accessibility and challenges of developing interfaces with UI frameworks and libraries. Accessibility barriers in programming tools and UI frameworks can complicate the processes of writing UI code, debugging, testing, and collaborating with sighted colleagues. This paper presents recommendations grounded in empirical findings to improve the accessibility of frameworks and libraries.
-
ParamMacros: Creating UI Automation Leveraging End-User Natural Language Parameterization
ParamMacros is a PBD system where users specify a natural language question about a website. ParamMacros then leverages the DOM structure to help generalize and parameterize to similar natural language questions.
-
PuzzleMe: Leveraging Peer Assessment for In-Class Programming Exercises
A tool that enhances in-class programming exercises by (1) enabling students to write unit tests that can be shared with the rest of the class (and used to test other students' code), (2) enables peer code reviews where students are intelligently grouped with other students to perform code reviews. PuzzleMe is intended for introductory classes and also includes a design to make testing more lightweight.
-
Understanding Accessibility and Collaboration in Programming for People with Visual Impairments
A study into how programmers with visual impairments work in the context of programming teams. We describe how some collaborative practices can impede collaboration and required invisible work and articulation work on the part of the programmers with visual impairments.
-
Understanding the Challenges and Needs of Programmers Writing Web Automation Scripts
Two studies to better understand the challenges of writing web automation code (particularly the kinds of context and feedback that are helpful when writing web automation code).
-
CoCapture: Effectively Communicating UI Behaviors on Existing Websites by Demonstrating and Remixing
CoCapture is a tool for creating behavior mockups for websites in the later stage of UI development. Unlike most other prototyping tools, CoCapture assumes that a partially functional UI exists and allows designers to describe UI behaviors by creating mockups that modify the existing UI behaviors by demonstrating and remixing.
-
Think-Aloud Computing: Supporting Rich and Low-Effort Knowledge Capture
Introduces "think-aloud computing", an application of think-alouds where users are encourage to speak in order to capture process-related as they work. For example, it might capture design intent, to-do items, problems encountered, and more. We present a formative study of think-aloud computing and a prototype of a tool to support think-aloud computing.
-
FlowMatic: An Immersive Authoring Tool for Creating Interactive Scenes in Virtual Reality
FlowMatic allows users to users to specify interactive behaviors (how to respond to users) while immersed in Virtual Reality (VR). It adapts dataflow programming to allow programming primitives to be modified through direct manipulation and provides immediate feedback.
-
A Hybrid Crowd-Machine Workflow for Program Synthesis
BashOn introduces a crowdsourcing approach to help make program synthesis systems more robust, reliable, and trustworthy, and reduces the cost of downstream data collection for training a program synthesis system.
-
EdCode: Towards Personalized Support at Scale for Remote Assistance in CS Education
EdCode applies a semi-asychronous on-demand help seeking model in a learning setting, aiming towards provide more personalized support at scale.
-
Sifter: A Hybrid Workflow for Theme-based Video Curation at Scale
Sifter improves the video curation process by combining automated techniques with a human-powered two-stage pipeline that browses, selects, and reaches an agreement.
-
Callisto: Capturing the "Why" by Connecting Conversations with Computational Narratives
We designed Callisto, a Jupyter Notebook extension for better explaining messy notebooks. Callisto captures and stores contextual links between discussion messages and notebook elements with minimal effort from users. Callisto allows notebook readers to better understand the current notebook content and the overall problem-solving process that led to it, by making it possible to browse the discussions and code history relevant to any part of the notebook.
-
Explore, Create, Annotate: Designing Digital Drawing Tools with Visually Impaired People
A study to better understand how visually impaired people explore and create labeled tactile graphics.
-
Improving Crowd-Supported GUI Testing with Structural Guidance
Two techniques, interactive event-flow graphs and GUI-level guidance, that guide GUI testers to discover more test cases and avoid duplicate test cases.