Steve Oney

School of Information
University of Michigan
4552 Leinweber Building
2200 Hayward Street
Ann Arbor, MI 48109
http://from.so/
1 (734) 999–0246
soney@umich.edu

Education

09/2008 – 04/2015
Pittsburgh, PA
Carnegie Mellon University
PhD in Human-Computer Interaction
MS in Human-Computer Interaction
Advisors: Brad Myers and Joel Brandt
Committee: Scott Hudson and John Zimmerman
09/2003 – 08/2008
Cambridge, MA
Massachusetts Institute of Technology
MEng in Computer Science
SB in Computer Science
SB in Mathematics

Professional Experience

09/2022 – present
Ann Arbor, MI
School of Information, University of Michigan
Associate Professor
09/2022 – present
Ann Arbor, MI
Computer Science and Engineering, University of Michigan
Associate Professor (by courtesy)
09/2016 – 09/2022
Ann Arbor, MI
School of Information, University of Michigan
Assistant Professor
01/2017 – 09/2022
Ann Arbor, MI
Computer Science and Engineering, University of Michigan
Assistant Professor (by courtesy)
09/2015 – 09/2016
Ann Arbor, MI
School of Information, University of Michigan
Presidents Post-Doctoral Fellow
09/2008 – 04/2015
Pittsburgh, PA
Carnegie Mellon University (Human-Computer Interaction Institute)
Graduate student and researcher
03/2013 – 06/2013
San Francisco, CA
Advanced Technologies Labs, Adobe Systems
Research Intern
06/2011 – 09/2011
San Francisco, CA
Advanced Technologies Labs, Adobe Systems
Research Intern
05/2009 – 08/2009
San Jose, CA
IBM Research, Alamaden, Research Intern
Research Intern
01/2007 – 08/2008
Cambridge, MA
MIT Media Lab, Cognitive Machines Group
Researcher (M.Eng)

Publications

 

Note: The names of authors who were students at the time of publication are underlined (including my name if I was a student at the time of publication).

Labels: : best paper award    : honorable mention or other award
Approximate Acceptance Rates:  UIST: 22%, CHI: 23%, VL/HCC: 30%, CSCW: 25%, ICSE: 19%, IMX: 26%

Heavily-reviewed Journal Manuscripts (J)

J.4
(PDF)
April Wang, Yan Chen, John Chung, Christopher Brooks, and Steve Oney. (2021) PuzzleMe: Leveraging Peer Assessment for In-Class Programming Exercises. The ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW). Toronto, Canada.
* J.3
(PDF)
Mauli Pandey, Vaishnav Kameswaran, Hrishikesh Rao, Sile O'Modhrain, and Steve Oney. (2021) Understanding Accessibility and Collaboration in Programming for People with Visual Impairments. The ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW). Toronto, Canada.
* Recognition for Contribution to Diversity and Inclusion
J.2
(PDF)
April Wang, Anant Mittal, Christopher Brooks, and Steve Oney. (2019) How Data Scientists Use Computational Notebooks for Real-Time Collaboration. The ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW). Austin, TX, USA. Volume 3, Article No. 39
J.1
(PDF)
Steve Oney, Christopher Brooks, and Paul Resnick. (2018) Creating Guided Code Explanations with chat.codes. The ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW). New York, NY, USA. Volume 2, Article No. 131

Heavily-reviewed Conference Papers (C)

C.36
(PDF)
Lei Zhang, Jin Pan, Jacob Gettig, Steve Oney, and Anhong Guo. (2024) VRCopilot: Authoring 3D Layouts with Generative Models in VR. The ACM Symposium on User Interface Software and Technology (UIST). Pittsburgh, PA, USA.
C.35
(PDF)
Rebecca Krosnick and Steve Oney. (2024) ScrapeViz: Hierarchical Representations for Web Scraping Macros. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Liverpool, UK.
C.34
(PDF)
Ashley Zhang, Xiaohang Tang, Steve Oney, and Yan Chen. (2024) CFlow: Supporting Semantic Flow Analysis of Students' Code in Programming Problems at Scale. ACM Conference on Learning @ Scale (L@S). Atlanta, GA.
C.33
(PDF)
Mauli Pandey, Steve Oney, and Andrew Begel. (2024) Towards Inclusive Source Code Readability Based on the Preferences of Programmers with Visual Impairments. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Honolulu, Hawaiʻi.
C.32
(PDF)
Ashley Zhang, Yan Chen, and Steve Oney. (2023) RunEx: Augmenting Regular-Expression Code Search with Runtime Values. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Washington, D.C..
C.31
(PDF)
Ashley Zhang, Yan Chen, and Steve Oney. (2023) VizProg: Identifying Misunderstandings by Visualizing Students' Coding Progress. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Hamburg, Germany.
C.30
(PDF)
Lei Zhang, Ashutosh Agrawal, Steve Oney, and Anhong Guo. (2023) VRGit: A Version Control System for Collaborative Content Creation in Virtual Reality. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Hamburg, Germany.
C.29
(PDF)
April Wang, Andrew Head, Ashley Zhang, Steve Oney, and Christopher Brooks. (2023) Colaroid: A Literate Programming Approach for Authoring Explorable Multi-Stage Tutorials. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Hamburg, Germany.
C.28
(PDF)
Rebecca Krosnick and Steve Oney. (2022) ParamMacros: Creating UI Automation Leveraging End-User Natural Language Parameterization. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Rome, Italy.
C.27
(PDF)
Mauli Pandey, Sharvari Bondre, Sile O'Modhrain, and Steve Oney. (2022) Accessibility of UI Frameworks and Libraries for Programmers with Visual Impairments. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Rome, Italy.
C.26
(PDF)
Rebecca Krosnick and Steve Oney. (2021) Understanding the Challenges and Needs of Programmers Writing Web Automation Scripts. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). (Virtual/Online).
C.25
(PDF)
Rebecca Krosnick, Fraser Anderson, Justin Matejka, Steve Oney, Walter Lasecki, Tovi Grossman, and George Fitzmaurice. (2021) Think-Aloud Computing: Supporting Rich and Low-Effort Knowledge Capture. ACM Conference on Human Factors in Computing Systems (CHI). (Virtual/Online).
C.24
(PDF)
Yan Chen, Sang Won Lee, and Steve Oney. (2021) CoCapture: Effectively Communicating UI Behaviors on Existing Websites by Demonstrating and Remixing. ACM Conference on Human Factors in Computing Systems (CHI). (Virtual/Online).
C.23
(PDF)
Lei Zhang and Steve Oney. (2020) FlowMatic: An Immersive Authoring Tool for Creating Interactive Scenes in Virtual Reality. The ACM Symposium on User Interface Software and Technology (UIST). Minneapolis, MN, USA.
* C.22
(PDF)
Yan Chen, Gabriel Matute, April Wang, Jaylin Herskovitz, Sang Won Lee, Walter Lasecki, and Steve Oney. (2020) EdCode: Towards Personalized Support at Scale for Remote Assistance in CS Education. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Dunedin, New Zealand.
*Best Short Paper
C.21
(PDF)
Yan Chen, Jaylin Herskovitz, Walter Lasecki, and Steve Oney. (2020) A Hybrid Crowd-Machine Workflow for Program Synthesis. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Dunedin, New Zealand.
C.20
(PDF)
Yan Chen, Andrés Monroy-Hernández, Ian Wehrman, Steve Oney, Walter Lasecki, and Rajan Vaish. (2020) Sifter: A Hybrid Workflow for Theme-based Video Curation at Scale. The ACM International Conference on Interactive Media Experiences (IMX). Barcelona, Spain.
C.19
(PDF)
Yan Chen, Mauli Pandey, Jean Song, Walter Lasecki, and Steve Oney. (2020) Improving Crowd-Supported GUI Testing with Structural Guidance. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Honolulu, Hawai'i.
C.18
(PDF)
April Wang, Zihan Wu, Christopher Brooks, and Steve Oney. (2020) Callisto: Capturing the "Why" by Connecting Conversations with Computational Narratives. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Honolulu, Hawai'i.
C.17
(PDF)
Mauli Pandey, Hari Subramonyam, Brooke Sasia, Steve Oney, and Sile O'Modhrain. (2020) Explore, Create, Annotate: Designing Digital Drawing Tools with Visually Impaired People. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Honolulu, Hawai'i.
* C.16
(PDF)
Lei Zhang and Steve Oney. (2019) Studying the Benefits and Challenges of Immersive Dataflow Programming. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Memphis, TN, USA. pp 223-227
*Best Short Paper
C.15
(PDF)
Steve Oney, Rebecca Krosnick, Joel Brandt, and Brad Myers. (2019) Implementing Multi-Touch Gestures with Touch Groups and Cross Events. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Glasgow, Scotland. Paper No. 355
C.14
(PDF)
Steve Oney, Alan Lundgard, Rebecca Krosnick, Michael Nebeling, and Walter Lasecki. (2018) Arboretum and Arbility: Improving Web Accessibility Through a Shared Browsing Architecture. The ACM Symposium on User Interface Software and Technology (UIST). Berlin, Germany. pp 937-949
C.13
(PDF)
Shih-Chieh Lin, Chang-Hong Hsu, Walter Talamonti, Yunqi Zhang, Steve Oney, Jason Mars, and Lingjia Tang. (2018) Adasa: A Conversational In-Vehicle Digital Assistant for Advanced Driver Assistance Features. The ACM Symposium on User Interface Software and Technology (UIST). Berlin, Germany. pp 531-542
C.12
(PDF)
Rebecca Krosnick, Sang Won Lee, Walter Lasecki, and Steve Oney. (2018) Expresso: Building Responsive Interfaces with Keyframes. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Lisbon, Portugal. pp 39-47
C.11
(PDF)
Yan Chen, Sang Won Lee, Yin Xie, YiWei Yang, Walter Lasecki, and Steve Oney. (2017) Codeon: On-Demand Software Development Assistance. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Denver, CO, USA. pp 6220-6231
C.10
(PDF)
Xin Rong, Shiyan Yan, Steve Oney, Mira Dontcheva, and Eytan Adar. (2016) CodeMend: Assisting Interactive Programming with Bimodal Embedding. The ACM Symposium on User Interface Software and Technology (UIST). Tokyo, Japan. pp 247-258
C.9
(PDF)
Yan Chen, Steve Oney, and Walter Lasecki. (2016) Towards Providing On-Demand Expert Support for Software Developers. The ACM CHI Conference on Human Factors in Computing Systems (CHI). San Jose, CA, USA. pp 3192-3203
C.8
(PDF)
Steve Oney, Brad Myers, and Joel Brandt. (2014) InterState: A Language and Environment for Expressing Interface Behavior. The ACM Symposium on User Interface Software and Technology (UIST). Honolulu, HI, USA. pp 263-272
C.7
(PDF)
Steve Oney, Chris Harrison, Amy Ogan, and Jason Wiese. (2013) ZoomBoard: A Diminutive QWERTY Soft Keyboard Using Iterative Zooming for Ultra-Small Devices. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Paris, France. pp 2799-3002
C.6
(PDF)
Steve Oney, Brad Myers, and Joel Brandt. (2012) ConstraintJS: Programming Interactive Behaviors for the Web by Integrating Constraints and States. The ACM Symposium on User Interface Software and Technology (UIST). Cambridge, MA, USA. pp 229-238
C.5
(PDF)
Rahul Pandita, Xusheng Xiao, Hao Zhong, Tao Xie, Steve Oney, and Amit Paradkar. (2012) Inferring Method Specifications from Natural Language API Descriptions. The International Conference on Software Engineering (ICSE). Zürich, Switzerland. pp 815-825
C.4
(PDF)
Steve Oney and Joel Brandt. (2012) Codelets: Linking Interactive Documentation and Example Code in the Editor. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Austin, TX, USA. pp 2697-2706
C.3
(PDF)
Steve Oney, John Barton, Brad Myers, Tessa Lau, and Jeffrey Nichols. (2011) Playbook: Revision Control & Comparison for Interactive Mockups. International Symposium on End-User Development (IS-EUD). Torre Canne, Italy. pp 2697-2706
C.2
(PDF)
Kursat Ozenc, Miso Kim, John Zimmerman, Steve Oney, and Brad Myers. (2010) How to Support Designers in Getting Hold of the Immaterial Material of Software. The ACM Conference on Human Factors in Computing Systems (CHI). Atlanta, GA, USA. pp 2513-2522
C.1
(PDF)
Steve Oney and Brad Myers. (2009) FireCrystal: Understanding Interactive Behaviors in Dynamic Web Pages. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Corvallis, OR, USA. pp 105-108

Book Chapters (B)

B.2
(PDF)
Mauli Pandey, Sharvari Bondre, Vaishnav Kameswaran, Hrishikesh Rao, Sile O'Modhrain, and Steve Oney. (2023) UI Development in Mixed-Ability Software Engineering Teams. Equity, Diversity, and Inclusion in Software Engineering (EDI).
B.1
(PDF)
Brad Myers, Amy Ko, Chris Scaffidi, Steve Oney, YoungSeok Yoon, Kerry Chang, Mary Beth Kery, and Toby Li. (2016) Making End User Development More Natural. Advances in End User Development (EUD).

Refereed Posters (P), Workshops (W), Doctoral Consortiums (D), Demos (E), and Panels (A)

W.21
(PDF)
Jiacheng Zhang, Carl Fan, and Steve Oney. (2025) Understanding Challenges and Needs of Using AI in Web Automation Systems. Designing and Developing User Interfaces with AI: Advancing Tools, Workflows, and Practices (CHI2025-CompUI). Yokohama, Japan.
W.20
(PDF)
Ashley Zhang, Yinuo Yang, Maryam Arab, Yan Chen, and Steve Oney. (2025) Understanding And Guiding Student-AI Interaction In Future Programming Education. CHI 2025 Workshop on Augmented Educators and AI (CHI2025-AE&AI). Yokohama, Japan.
E.19
(PDF)
Yinuo Yang and Steve Oney. (2024) VizCode: A Practical Real-time Tool for In-Class Computer Programming Tutoring. Demos at the ACM Conference on Learning @ Scale (L@S Demos). Atlanta, GA.
E.18
(PDF)
Ashley Zhang, Xiaohang Tang, Steve Oney, and Yan Chen. (2024) Demonstration of CFlow: Supporting Semantic Flow Analysis of Students' Code in Programming Problems at Scale. Demos at the ACM Conference on Learning @ Scale (L@S Demos). Atlanta, GA.
A.17
(PDF)
Jane Im, Himanshu Zade, Steve Oney, Pamela Wisniewski, and Kentaro Toyama. (2024) Improving Advising Relationships Between PhD Students and Faculty in Human-Computer Interaction. Extended Abstracts of the ACM CHI Conference on Human Factors in Computing Systems (CHI EA). Honolulu, Hawaiʻi.
W.16
(PDF)
April Wang, Zihan Wu, Christopher Brooks, and Steve Oney. (2024) "Don't Step on My Toes": Resolving Editing Conflicts in Real-Time Collaboration in Computational Notebooks. The IDE Workshop (IDE Workshop). Lisbon, Portugal.
W.15
(PDF)
Rebecca Krosnick and Steve Oney. (2023) Promises and Pitfalls of Using LLMs for Scraping Web UIs. CHI 2023 Workshop: Future of Computational Approaches for Understanding & Adapting User Interfaces (CHI2023-CompUI). Hamburg, Germany.
P.14
(PDF)
Shiyu Xu, Ashley Zhang, and Steve Oney. (2023) How Pairing by Code Similarity Influences Discussions in Peer Learning. Late-Breaking Work at The ACM CHI Conference on Human Factors in Computing Systems (CHI LBW). Hamburg, Germany.
P.13
(PDF)
April Wang, Dakuo Wang, Jaimie Drozdal, Xuye Liu, Soya Park, Steve Oney, and Christopher Brooks. (2021) What Makes a Well-Documented Notebook? A Case Study of Data Scientists’ Documentation Practices in Kaggle. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA). (Virtual/Online).
W.12
(PDF)
Yan Chen, Jasmine Jones, and Steve Oney. (2020) On-Demand Collaboration in Programming. Microsoft New Future of Work Symposium (MSNFWS). (Virtual/Online).
W.11
(PDF)
Rebecca Krosnick and Steve Oney. (2020) Toward Providing Live Feedback in Web Automation IDEs. Workshop on Live Programming (LIVE). (Virtual/Online).
P.10
(PDF)
Mauli Pandey, Michael Nebeling, Sun Young Park, and Steve Oney. (2019) Exploring the Tracking Needs and Practices of Recreational Athletes.. EAI International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth). Trento, Italy.
W.9
(PDF)
April Wang, Steve Oney, and Christopher Brooks. (2019) Redesigning Notebooks for Data Science Education. Human-Centered Study of Data Science Work Practices Workshop, ACM Conference on Human Factors in Computing Systems (CHI) (HuSDat). Glasgow, Scotland.
W.8
(PDF)
Louis Spinelli, Mauli Pandey, and Steve Oney. (2018) Attention Patterns for Code Animations: Using Eye Trackers to Evaluate Dynamic Code Presentation Techniques. Programming Experience (PX/18). Nice, France.
W.7
(PDF)
Yan Chen, Steve Oney, and Walter Lasecki. (2016) Expert Crowd Support Systems for Software Developers. Collective Intelligence (CI). New York, NY, USA.
W.6
(PDF)
Steve Oney, Brad Myers, and Joel Brandt. (2013) Euclase: A Live Development Environment with Constraints and FSMs. International Workshop on Live Programming at ICSE (LIVE). San Francisco, CA, USA.
W.5
(PDF)
Brad Myers, Steve Oney, YoungSeok Yoon, and Joel Brandt. (2013) Creativity Support in Authoring and Backtracking. CHI 2013 Workshop on Evaluation Methods for Creativity Support Environments. Paris, France.
D.4
(PDF)
Steve Oney. (2011) Development Tools for Interactive Behaviors. Doctoral Consortium: IS-EUD (IS-EUD DC). Torre Canne, Italy.
D.3
(PDF)
Steve Oney. (2010) Democratizing Computational Tools for Interaction Designers. Graduate Consortium: VL/HCC (VL/HCC GC). Madrid, Spain.
D.2
(PDF)
Steve Oney. (2009) Empowering Designers with Creativity Support Tools. Graduate Consortium: VL/HCC (VL/HCC GC). Corvallis, OR, USA.
W.1
(PDF)
Steve Oney, Brad Myers, and John Zimmerman. (2009) Visions for Euclase: Ideas for Supporting Creativity through Better Prototyping of Behaviors. Workshop on Computational Creativity (CHI). Boston, MA, USA.

Preprints (R)

R.1
(PDF)
Steve Oney. (2024) EDBooks: AI-Enhanced Interactive Narratives for Programming Education. null ((Preprint)).

Theses (T)

T.2
(PDF)
Steve Oney. (2015) Expressing Interactivity with States and Constraints. Ph.D. Thesis (CMU). Pittsburgh, PA.
T.1
(PDF)
Steve Oney. (2008) Natural Language Search of Structured Documents. M.eng Thesis (MIT). Cambridge, MA, USA.

Grants

11/2020
$10,000

Steve Oney (PI)

Adobe, Inc.
10/2020
$60,000
Designing and Building Collaborative Tools for Mixed-Ability Programming Teams

Steve Oney (PI) (and Mauli Pandey, unofficially)

Google Inc.
Award for Inclusion Research
09/2020
$499,209
Improving Web Accessibility Through Multi-Resolution Mixed-Initiative Interaction Tools

Steve Oney (PI)

National Science Foundation (NSF)
IIS: Cyber-Human Systems (CHS)
07/2019
$10,875
Applying Literate Programming Approaches to Support Semantic Annotation

Andrea Thomer (PI) and Steve Oney (co-PI)

The U-M Office of Research (UMOR)
10/2019
$598,926
Scalable Remote Peer Help for Programming Education

Steve Oney (PI), Paul Resnick, and Christopher Brooks

National Science Foundation (NSF)
Improving Undergraduate STEM Education: Education and Human Resources (IUSE: EHR)
01/2018
$174,981
Designing Scalable Help Tools for Programming Courses

Steve Oney (PI)

National Science Foundation (NSF)
Cyber-Human Systems (CHS) CRII
11/2017
$37,000
Prototyping Tools to Improve Crowd Based Training for IVA Development

Steve Oney (PI) and Walter Lasecki

Clinc, Inc.
04/2017
$198,327
End-User Techniques for Aggregating and Analyzing Exercise and Physical Data

Steve Oney (PI), Michael Nebeling, and Sun Young Park

Michigan Exercise Science & Sports Initiative

Awards

 
Note: Does not include best paper awards or nominations (in Publications above)
09/2021
UMSI Excellence in Instruction Award
09/2015
University of Michigan President's Postdoctoral Fellowship
10/2009
UIST Student Innovation Contest, 1st place
Part of winning team in most creative category
09/2009
Google/UNCF Scholarship
One-year scholarship for $10,000
09/2009 – 05/2012
Ford Foundation Predoctoral Fellowship
Annual stipend of $20,000 for three years, awarded to 60 doctoral students nationwide across disciplines
09/2008 – 05/2011
ARCS Foundation Scholarship (Pittsburgh Chapter)
Annual stipend of $5,000 for three years. Awarded to 13 doctoral students in the Pittsburgh area (CMU & University of Pittsburgh)
09/2008
MIT Batttlecode Open Programming Competition Finalist
09/2008
NEWMAC Academic All-Conference
Awarded for academic success while a member of MIT's varsity track team

Invited Presentations

11/2021
(remote)
Stanford HCI Seminar
Designing Tools for Remote Communication in Programming
11/2021
(remote)
Harvard Computer Science Colloquium
Designing Tools for Remote Communication in Programming
5/2021
(remote)
Adobe Research HCI Seminar
Designing Tools for Remote Communication in Programming
12/2020
(remote)
University of Washington DUB Seminar
Designing Tools for Remote Communication in Programming
10/2020
(remote)
MIT CSAIL HCI Seminar
Designing Tools for Remote Communication in Programming
11/2019
Bloomington, IN
Indiana University
Designing Tools for Remote Communication and Collaboration
11/2018
Williamstown, MA
Williams College
CS Colloquium—Designing Tools for More Effective Remote Communication
11/2017
Madison, WI
University of Wisconsin
HCI Seminar: Designing Tools for Remote Communication Between Programmers
03/2016
Ann Arbor, MI
University of Michigan School of Information
Programming Tools that Speak our Language
10/2015
South Bend, IN
University of Notre Dame Department of Computer Science and Engineering
Expressing Interactivity with States and Constraints
04/2015
Chicago, IL
University of Illinois at Chicago Department of Computer Science
Expressing Interactivity with States and Constraints
03/2015
Boston, MA
Boston University Department of Computer Science
Expressing Interactivity with States and Constraints
03/2015
Palo Alto, CA
FX Palo Alto Laboratory
Expressing Interactivity with States and Constraints
03/2015
Stony Brook, CA
Stony Brook Computer Science Department
Expressing Interactivity with States and Constraints
02/2015
Irvine, CA
University of California at Irvine
Expressing Interactivity with States and Constraints
03/2010
Dagstuhl, Germany
Dagstuhl: Practical Software Testing: Tool Automation and Human Factors
06/2009
San Jose, CA
IBM Almaden Lunch Seminar
FireCrystal: Understanding Interactive Behaviors in Dynamic Web Pages

Service

Organizing Committee

2023 – 2027
Steering Committee
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)
2026
General Chair (with Michael Nebeling)
UIST 2026
2024 – 2025
Most Influential Paper Award Co-Chair
VL/HCC 2024 and 2025
2021
Program Co-Chair (with Jácome Miguel Costa Cunha)
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)
2020
Graduate Consortium co-chair (with Michael Lee)
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)

Program Committee

2017 – present
ACM Symposium on User Interface Software and Technology (UIST)
2022
ACM Conference On Computer-Supported Cooperative Work And Social Computing (CSCW)
2016 – present
ACM Conference on Human Factors in Computing Systems (CHI)
2018, 2020
ACM International Conference on Supporting Group Work (GROUP)
2017 – 2020
Programming Experience Workshop (PX)
2019
International Workshop on Eye Movements in Programming (EMIP)
2019
ACM Conference on Tangible, Embedded, and Embodied Interactions (TEI)
2019
Tech Notes for the ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS)

Peer Reviewing

2018
IEEE Transactions of Software Engineering (TSE)
2018
ACM Transactions on Computer-Human Interaction (TOCHI)
2010 – present
ACM Conference on Human Factors in Computing Systems (CHI)
2014
Conference on Human-Computer Interaction with Mobile Devices (MobileHCI)
2010, 2012
ACM Conference on Designing Interactive Systems (DIS)
2008, 2009
Philippine Journal of Science (PJS)
2013 – 2015
ACM UIST Operations Committee (mobile program guide)

Operations Committee

2011 – 2015
ACM Symposium on User Interface Software & Technology (UIST)
2012 – 2015
ACM CHI Operations Committee (mobile program guide)
2013 – 2015
ACM ITS Operations Committee (mobile program guide)
2012
ACM UbiComp Committee (mobile program guide)
2010, 2011
ACM CHI student volunteer (mobile guide development team)

UMSI

2023 – (present)
UMSI Culture and Community Committee Co-Chair
2018 – 2020 & 2021 – 2022
School of Information BSI Committee
2021 – present
Faculty Advisor for UMSI chapter of Upsilon Pi Epsilon (International Honor Society for the Computing and Information Disciplines)
2019 – 2020
School of Information Technical Curriculum Task Force
2018 – 2019
University of Michigan Interactive and Social Computing (MISC) Coordinator

Other

2009 – 2015
CMU Computer Science outreach roadshow volunteer (with Women@SCS & SCS4All)
2011 – 2014
CMU Human-Computer Interaction Institute (HCII) ombudsperson
2010
CMU Human-Computer Interaction Institute (HCII) visit weekend co-chair
2009, 2010
CMU Human-Computer Interaction Institute (HCII) PhD lunch coordinator
2025
CSE Promotion and Tenure Committee

Teaching

2023 – present
University of Michigan
Instructor (and creator) – SI 379 (Building Interactive Applications)
2021 – 2023
University of Michigan
Instructor (and creator) – SI 579 (Building Interactive Applications)
2016 – present
University of Michigan
Instructor – SI 106 (Programs, Information, & People)
2019
University of Michigan & Coursera
Python 3 Programming Specialization

https://www.coursera.org/specializations/python-3-programming

Fall 2012
Carnegie Mellon
Instructor – Web Lab, Programming User Interfaces

Developed syllabus, wrote lectures, created projects, presented, graded, and held office hours weekly. Instructor rating: 4.7/5.0

Fall 2010
Carnegie Mellon
Instructor – GUI Lab, Programming User Interfaces

Developed syllabus, wrote lectures, created projects, presented, graded, and held office hours weekly. Instructor rating: 4.6/5.0

Fall 2007 & Spring 2008
MIT
Instructor – GUI Lab, Programming User Interfaces

Taught three recitation sections per week, held weekly office hours, and graded students’ exams

Summer 2007
MIT
Teaching Assistant – Interphase Physics I

Taught three classes per week, held weekly office hours, and mentored a group of incoming MIT freshmen

Spring 2005
MIT
Teaching Assistant – Technology Enabled Learning (TEAL) Physics II
Spring 2005 – Fall 2006
MIT
Laboratory Assistant – Circuits and Electronics
Fall 2006
MIT
Laboratory Assistant – Computational Structures

Students Supervised

Dissertation Chair (Ph.D. Graduates)

Fall 2015 – Fall 2020
University of Michigan
Yan Chen (School of Information)
On-Demand Collaboration in Programming
Currently: Assistant Professor at Virginia Tech
Fall 2018 – Spring 2024
University of Michigan
Lei Zhang (School of Information)
Designing End-user Creation Tools for Immersive Experiences
(co-advised with Anhong Guo)
Currently: Assistant Professor at NJIT
Fall 2018 – Summer 2023
University of Michigan
(Mauli) Maulishree Pandey (School of Information)
Accessibility of Collaborative Programming for Blind and Visually Impaired Developers
Currently: UX Researcher at Google
Fall 2018 – Spring 2023
University of Michigan
(April) Yi Wang (School of Information)
Interactive Programming Interfaces for Data Science Collaboration and Learning
(co-advised with Christopher Brooks)
Currently: Assistant Professor at ETH Zürich
Fall 2017 – Summer 2023
University of Michigan
Rebecca Krosnick (Computer Science and Engineering)
Improving Web Automation Tools through UI Context and Demonstration
Currently: Software Engineer at Postman

Ph.D. Advisees (Ongoing)

Fall 2021 – present
University of Michigan
(Ashley) Ge Zhang (School of Information)
Fall 2023 – present
University of Michigan
Jiacheng Zhang (Scool of Information)

Thesis Committees (Ph.D.)

2021
University of Michigan
Hari Subramonyam (School of Information)
Designing AI Experiences: Boundary Representations, Collaborative Processes, and Data Tools
2020
University of Michigan
Vaspol Ruamviboonsuk (Computer Science and Engineering)
Understanding and Improving the Performance of Web Page Loads
2019
University of Michigan
Shih-Chieh Lin (Computer Science and Engineering)
Cross-Layer System Design for Autonomous Driving
2017
University of Michigan
Sang Won Lee (Computer Science and Engineering)
Improving User Involvement Through Live, Collaborative Creation
2017
University of Michigan
Xin Rong (School of Information)
Neural Language Models for Data-Driven Programming Support
2023
University of Michigan
David Moon (Computer Science and Engineering)
2025
University of Michigan
Zihan Wu
Designing and Evaluating Fine-Grained Interactive Practice Tools for Novice Programming Learners
2025
University of Michigan
Jingyuan Zhu (Computer Science and Engineering)

Thesis Chair (M.S.)

2018
University of Michigan
(Mauli) Maulishree Pandey (School of Information)
Exploring and Designing for the Self-Tracking Needs of Recreational Athletes
2024
University of Michigan
KJ Green
2024
University of Michigan
Yue Shen
An LLM-based Method for Tailoring Coding Tutorials
2024
University of Michigan
Carl Fan

Thesis Committees (M.S.)

2020
University of Michigan
Xunan (Andy) Zhou (School of Information)
Conversational Agent Experience: What makes a good skill?
2020
University of Michigan
Kangning Chen (School of Information)
Providing Examples and Tool Support for Novice AR Creators
2019
University of Michigan
Katy Madier (School of Information)
Enabling Low-cost Co-located Virtual Reality Experiences
2024
University of Michigan
Jiaqi Wu
Exploring Bridges Between Creative Coding and Visual Generative AI

Thesis Chair (Undergraduate)

2024
University of Michigan
Yinuo Yang
 

Press

VentureBeat, 2014
Adobe and CMU researchers unveil a brilliant new JavaScript library: ConstraintJS
June 23
Wired, 2013
Researchers Figure Out How You Can Type on a Smartwatch
May 1
Slashdot, 2013
CMU Offers Wee QWERTY Texting Tech for Impossibly Tiny Devices
May 1
Gizmodo, 2013
How Typing on a Smart Watch Might Actually Make Sense
April 29
MIT Tech Review, 2013
A QWERTY Keyboard for your Wrist
April 27

Patent

11/2016
US Patent number 9,495,134. "Methods and Apparatus for Code Segment Handling" Brandt, J. & Oney, S.