Steve Oney

A mixed-initiative tool for immersive 3D layout design in VR. It integrates generative AI to assist users in co-creating room layouts using voice commands, multimodal interactions, and wireframe-based representations. VRCopilot supports three modes: manual creation, automatic AI generation, and scaffolded creation with wireframes, balancing user control and automation.

A programming-by-demonstration tool that helps users create and visualize web scraping macros across multiple pages. ScrapeViz offers a visual storyboard of scraping behaviors, making it easier to understand, trace data sources, and identify errors.

Introduces CFlow, a system for visualizing and analyzing large numbers of student code submissions to help instructors understand common mistakes and patterns. CFlow uses semantic labeling and code structure analysis to cluster code lines across submissions and present them in a visualization that resembles a single code sample, allowing instructors to see the overall semantic flow while still being able to dive into individual submissions.

Explores the code readability preferences of blind and visually impaired (BVI) developers and how they differ from sighted developers, through a qualitative study. Contributes an inclusive taxonomy for code readability and recommendations for programming tools, code standards, and languages to better support BVI developers.

A system that improves code search by augmenting regular expressions with the ability to specify constraints on runtime values of specific expressions.

VizProg is a tool that allows instructors of large programming classes to monitor students' progress at scale. It represents students' code in a 2D map where dots representing students move further to the right as they get closer to a correct solution. VizProg uses vertical space to represent different approaches to solving the problem (so students towards the bottom of the map are using a different approach than students at the top of the map).

Colaroid is an extension to the Visual Studio Code IDE that makes it easier to create mutli-step tutorials about a large code sample. Developers can write their code in the IDE and as they are writing their code, Colaroid allows them to divide their work into steps. Colaroid tutorials are augmented computational notebooks, where snippets and outputs represent a snapshot of a project, with source code differences highlighted, complete source code context for each snippet, and the ability to load and tinker with any stage of the project in a linked IDE.

A version control system designed especially for Virtual Reality (VR). VRGit allows users to view and navigate the version history of a 3-D scene while immersed in VR. It also allows users to share and discuss history graphs collaboratively.

ParamMacros is a PBD system where users specify a natural language question about a website. ParamMacros then leverages the DOM structure to help generalize and parameterize to similar natural language questions.

Two studies into the accessibility and challenges of developing interfaces with UI frameworks and libraries. Accessibility barriers in programming tools and UI frameworks can complicate the processes of writing UI code, debugging, testing, and collaborating with sighted colleagues. This paper presents recommendations grounded in empirical findings to improve the accessibility of frameworks and libraries.

A tool that enhances in-class programming exercises by (1) enabling students to write unit tests that can be shared with the rest of the class (and used to test other students' code), (2) enables peer code reviews where students are intelligently grouped with other students to perform code reviews. PuzzleMe is intended for introductory classes and also includes a design to make testing more lightweight.

A study into how programmers with visual impairments work in the context of programming teams. We describe how some collaborative practices can impede collaboration and required invisible work and articulation work on the part of the programmers with visual impairments.

Two studies to better understand the challenges of writing web automation code (particularly the kinds of context and feedback that are helpful when writing web automation code).

Introduces "think-aloud computing", an application of think-alouds where users are encourage to speak in order to capture process-related as they work. For example, it might capture design intent, to-do items, problems encountered, and more. We present a formative study of think-aloud computing and a prototype of a tool to support think-aloud computing.

CoCapture is a tool for creating behavior mockups for websites in the later stage of UI development. Unlike most other prototyping tools, CoCapture assumes that a partially functional UI exists and allows designers to describe UI behaviors by creating mockups that modify the existing UI behaviors by demonstrating and remixing.

FlowMatic allows users to users to specify interactive behaviors (how to respond to users) while immersed in Virtual Reality (VR). It adapts dataflow programming to allow programming primitives to be modified through direct manipulation and provides immediate feedback.

BashOn introduces a crowdsourcing approach to help make program synthesis systems more robust, reliable, and trustworthy, and reduces the cost of downstream data collection for training a program synthesis system.

EdCode applies a semi-asychronous on-demand help seeking model in a learning setting, aiming towards provide more personalized support at scale.

Sifter improves the video curation process by combining automated techniques with a human-powered two-stage pipeline that browses, selects, and reaches an agreement.

Two techniques, interactive event-flow graphs and GUI-level guidance, that guide GUI testers to discover more test cases and avoid duplicate test cases.

We designed Callisto, a Jupyter Notebook extension for better explaining messy notebooks. Callisto captures and stores contextual links between discussion messages and notebook elements with minimal effort from users. Callisto allows notebook readers to better understand the current notebook content and the overall problem-solving process that led to it, by making it possible to browse the discussions and code history relevant to any part of the notebook.

A study to better understand how visually impaired people explore and create labeled tactile graphics.

We reported how synchronous editing in computational notebooks changes the way data scientists work together compared to working on individual notebooks through a formative survey and an observational study. Working on the synchronous notebooks improves collaboration by creating a shared context, encouraging more exploration, and reducing communication costs. However, the current synchronous editing features may lead to unbalanced participation and activity interference without strategic coordination.

In this paper, we study the benefits and challenges of immersive dataflow authoring, a paradigm that allows users to build VR applications using dataflow notation while immersed in the VR world.

Proposes two primitives to improve multi-touch programming: touch groups (which summarize multiple fingers rather than tracking them manually) and cross events (which summarize the movement of touch groups).

Introduces three features to improve textual-based conversations about code: deictic code references (where a message points to a specific region of code), in-line code diffs (to show how code changed throughout a conversation), and conversation version tracking (which allows viewers to view the state of code as it evolves through out the conversation).

A conversational digital assistant to help drivers use and understand Advanced Driver Assistance Systems (ADAS).

A system that helps visually impaired web users overcome accessibility barriers through targeted handoffs of tasks that require visual-spatial interaction.

A programming-by-demonstration approach for creating responsive UIs, where users create keyframes at a few representative viewport sizes and specify smooth or jump (i.e., linear interpolation) transitions between them.

CodeOn enables effective task hand-offs between developers and remote helpers by allowing asynchronous responses to on-demand requests.

A system that helps programmers find appropriate API functions and parameter values from natural language.

Two studies that present the opportunities and the design recommendations of on-demand remote support systems for developers.

A system that helps programmers use examples from the web better by embedding interactive explanations and cutomizers in the programmer's code editor.

FireCrystal is (was) a Firefox extension for inspecting dynamic behaviors in web pages. When a developer wants to understand how some dynamic behavior in a page is implemented, FireCrystal helps them by identifying the code that runs in between DOM changes.