Steve Oney

School of Information
University of Michigan
4381 North Quadrangle
105 South State Street
Ann Arbor, MI 48109
http://from.so/
1 (734) 999–0246
soney@umich.edu

Education

09/2008 – 04/2015
Pittsburgh, PA
Carnegie Mellon University
PhD in Human-Computer Interaction
MS in Human-Computer Interaction
Advisors: Brad Myers and Joel Brandt
Committee: Scott Hudson and John Zimmerman
09/2003 – 08/2008
Cambridge, MA
Massachusetts Institute of Technology
MEng in Computer Science
SB in Computer Science
SB in Mathematics

Professional Experience

09/2016 – present
Ann Arbor, MI
School of Information, University of Michigan
Assistant Professor
01/2017 – present
Ann Arbor, MI
Computer Science and Engineering, University of Michigan
Assistant Professor (by courtesy)
09/2015 – 09/2016
Ann Arbor, MI
School of Information, University of Michigan
Presidents Post-Doctoral Fellow
09/2008 – 04/2015
Pittsburgh, PA
Carnegie Mellon University (Human-Computer Interaction Institute)
Graduate student and researcher
03/2013 – 06/2013
San Francisco, CA
Advanced Technologies Labs, Adobe Systems, Research Intern
Research Intern
06/2011 – 09/2011
San Francisco, CA
Advanced Technologies Labs, Adobe Systems
Research Intern
05/2009 – 08/2009
San Jose, CA
IBM Research, Alamaden, Research Intern
Research Intern
01/2007 – 08/2008
Cambridge, MA
MIT Media Lab, Cognitive Machines Group
Researcher (M.Eng)

Publications

Note: The names of authors who were students at the time of publication are underlined (including my name if I was a student at the time of publication).

Labels: : best paper award    : honorable mention
Approximate Acceptance Rates:  UIST: 22%, CHI: 23%, VL/HCC: 30%, CSCW: 25%, ICSE: 19%, IMX: 26%

Heavily-reviewed Conference Papers (C) and Journal Manuscripts (J)

C.25
(PDF)
Lei Zhang and Steve Oney. (2020) FlowMatic: An Immersive Authoring Tool for Creating Interactive Scenes in Virtual Reality. The ACM Symposium on User Interface Software and Technology (UIST). Minneapolis, MN, USA.
* C.24
(PDF)
Yan Chen, Gabriel Matute, April Wang, Jaylin Herskovitz, Sang Won Lee, Walter Lasecki, and Steve Oney. (2020) EdCode: Towards Personalized Support at Scale for Remote Assistance in CS Education. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Dunedin, New Zealand.
*Best Short Paper
C.23
(PDF)
Yan Chen, Jaylin Herskovitz, Walter Lasecki, and Steve Oney. (2020) A Hybrid Crowd-Machine Workflow for Program Synthesis. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Dunedin, New Zealand.
C.22
(PDF)
Yan Chen, Andrés Monroy-Hernández, Ian Wehrman, Steve Oney, Walter Lasecki, and Rajan Vaish. (2020) Sifter: A Hybrid Workflow for Theme-based Video Curation at Scale. The ACM International Conference on Interactive Media Experiences (IMX). Barcelona, Spain.
C.21
(PDF)
April Wang, Zihan Wu, Christopher Brooks, and Steve Oney. (2020) Callisto: Capturing the "Why" by Connecting Conversations with Computational Narratives. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Honolulu, Hawai'i. (to appear)
C.20
(PDF)
Yan Chen, Mauli Pandey, Jean Song, Walter Lasecki, and Steve Oney. (2020) Improving Crowd-Supported GUI Testing with Structural Guidance. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Honolulu, Hawai'i. (to appear)
C.19
(PDF)
Mauli Pandey, Hari Subramonyam, Steve Oney, and Sile O'Modhrain. (2020) Explore, Create, Annotate: Designing Digital Drawing Tools with Visually Impaired People. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Honolulu, Hawai'i. (to appear)
J.18
(PDF)
April Wang, Anant Mittal, Christopher Brooks, and Steve Oney. (2019) How Data Scientists Use Computational Notebooks for Real-Time Collaboration. The ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW). Austin, TX, USA. Volume 3, Article No. 39
* C.17
(PDF)
Lei Zhang and Steve Oney. (2019) Studying the Benefits and Challenges of Immersive Dataflow Programming. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Memphis, TN, USA. pp 223-227
*Best Short Paper
C.16
(PDF)
Steve Oney, Rebecca Krosnick, Joel Brandt, and Brad Myers. (2019) Implementing Multi-Touch Gestures with Touch Groups and Cross Events. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Glasgow, Scotland. Paper No. 355
J.15
(PDF)
Steve Oney, Christopher Brooks, and Paul Resnick. (2018) Creating Guided Code Explanations with chat.codes. The ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW). New York, NY, USA. Volume 2, Article No. 131
C.14
(PDF)
Shih-Chieh Lin, Chang-Hong Hsu, Walter Talamonti, Yunqi Zhang, Steve Oney, Jason Mars, and Lingjia Tang. (2018) Adasa: A Conversational In-Vehicle Digital Assistant for Advanced Driver Assistance Features. The ACM Symposium on User Interface Software and Technology (UIST). Berlin, Germany. pp 531-542
C.13
(PDF)
Steve Oney, Alan Lundgard, Rebecca Krosnick, Michael Nebeling, and Walter Lasecki. (2018) Arboretum and Arbility: Improving Web Accessibility Through a Shared Browsing Architecture. The ACM Symposium on User Interface Software and Technology (UIST). Berlin, Germany. pp 937-949
C.12
(PDF)
Rebecca Krosnick, Sang Won Lee, Walter Lasecki, and Steve Oney. (2018) Expresso: Building Responsive Interfaces with Keyframes. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Lisbon, Portugal. pp 39-47
C.11
(PDF)
Yan Chen, Sang Won Lee, Yin Xie, YiWei Yang, Walter Lasecki, and Steve Oney. (2017) Codeon: On-Demand Software Development Assistance. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Denver, CO, USA. pp 6220-6231
C.10
(PDF)
Xin Rong, Shiyan Yan, Steve Oney, Mira Dontcheva, and Eytan Adar. (2016) CodeMend: Assisting Interactive Programming with Bimodal Embedding. The ACM Symposium on User Interface Software and Technology (UIST). Tokyo, Japan. pp 247-258
C.9
(PDF)
Yan Chen, Steve Oney, and Walter Lasecki. (2016) Towards Providing On-Demand Expert Support for Software Developers. The ACM CHI Conference on Human Factors in Computing Systems (CHI). San Jose, CA, USA. pp 3192-3203
C.8
(PDF)
Steve Oney, Brad Myers, and Joel Brandt. (2014) InterState: A Language and Environment for Expressing Interface Behavior. The ACM Symposium on User Interface Software and Technology (UIST). Honolulu, HI, USA. pp 263-272
C.7
(PDF)
Steve Oney, Chris Harrison, Amy Ogan, and Jason Wiese. (2013) ZoomBoard: A Diminutive QWERTY Soft Keyboard Using Iterative Zooming for Ultra-Small Devices. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Paris, France. pp 2799-3002
C.6
(PDF)
Brad Myers, Steve Oney, and Joel Brandt. (2012) ConstraintJS: Programming Interactive Behaviors for the Web by Integrating Constraints and States. The ACM Symposium on User Interface Software and Technology (UIST). Cambridge, MA, USA. pp 229-238
C.5
(PDF)
Rahul Pandita, Tao Xie, Hao Zhong, Xusheng Xiao, Steve Oney, and Amit Paradkar. (2012) Inferring Method Specifications from Natural Language API Descriptions. The International Conference on Software Engineering (ICSE). Zürich, Switzerland. pp 815-825
C.4
(PDF)
Steve Oney and Joel Brandt. (2012) Codelets: Linking Interactive Documentation and Example Code in the Editor. The ACM CHI Conference on Human Factors in Computing Systems (CHI). Austin, TX, USA. pp 2697-2706
C.3
(PDF)
Steve Oney, Tessa Lau, John Barton, Jeffrey Nichols, and Brad Myers. (2011) Playbook: Revision Control & Comparison for Interactive Mockups. International Symposium on End-User Development (IS-EUD). Torre Canne, Italy. pp 2697-2706
C.2
(PDF)
Kursat Ozenc, Brad Myers, Miso Kim, John Zimmerman, and Steve Oney. (2010) How to Support Designers in Getting Hold of the Immaterial Material of Software. The ACM Conference on Human Factors in Computing Systems (CHI). Atlanta, GA, USA. pp 2513-2522
C.1
(PDF)
Steve Oney and Brad Myers. (2009) FireCrystal: Understanding Interactive Behaviors in Dynamic Web Pages. IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Corvallis, OR, USA. pp 105-108

Book Chapters (B)

B.1
(PDF)
Brad Myers, Amy Ko, Chris Scaffidi, Steve Oney, YoungSeok Yoon, Kerry Chang, Mary Beth Kery, and Toby Li. (2016) Making End User Development More Natural. Advances in End User Development.

Refereed Posters (P), Workshops (W), and Doctoral Consortiums (D)

W.11
Yan Chen, Jasmine Jones, and Steve Oney. (2020) On-Demand Collaboration in Programming. Microsoft New Future of Work Symposium. (Virtual/Online).
P.10
(PDF)
Mauli Pandey, Steve Oney, Michael Nebeling, and Sun Young Park. (2019) Exploring the Tracking Needs and Practices of Recreational Athletes.. EAI International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth). Trento, Italy.
W.9
(PDF)
April Wang, Steve Oney, and Christopher Brooks. (2019) Redesigning Notebooks for Data Science Education. Human-Centered Study of Data Science Work Practices Workshop, ACM Conference on Human Factors in Computing Systems (CHI) (HuSDat). Glasgow, Scotland.
W.8
(PDF)
Louis Spinelli, Mauli Pandey, and Steve Oney. (2018) Attention Patterns for Code Animations: Using Eye Trackers to Evaluate Dynamic Code Presentation Techniques. Programming Experience (PX/18). Nice, France.
W.7
(PDF)
Yan Chen, Steve Oney, and Walter Lasecki. (2016) Expert Crowd Support Systems for Software Developers. Collective Intelligence (CI). New York, NY, USA.
W.6
(PDF)
Steve Oney, Brad Myers, and Joel Brandt. (2013) Euclase: A Live Development Environment with Constraints and FSMs. International Workshop on Live Programming at ICSE (LIVE). San Francisco, CA, USA.
W.5
(PDF)
Brad Myers, Joel Brandt, Steve Oney, and YoungSeok Yoon. (2013) Creativity Support in Authoring and Backtracking. Workshop on Evaluation Methods for Creativity Support Environments, CHI. Paris, France.
D.4
(PDF)
Steve Oney. (2011) Development Tools for Interactive Behaviors. Doctoral Consortium: IS-EUD. Torre Canne, Italy.
D.3
(PDF)
Steve Oney. (2010) Democratizing Computational Tools for Interaction Designers. Graduate Consortium: VL/HCC. Madrid, Spain.
D.2
(PDF)
Steve Oney. (2009) Empowering Designers with Creativity Support Tools. Graduate Consortium: VL/HCC. Corvallis, OR, USA.
W.1
(PDF)
Steve Oney, Brad Myers, and John Zimmerman. (2009) Visions for Euclase: Ideas for Supporting Creativity through Better Prototyping of Behaviors. Workshop on Computational Creativity (CHI). Boston, MA, USA.

Theses (T)

T.2
(PDF)
Steve Oney. (2015) Expressing Interactivity with States and Constraints. Ph.D. Thesis (CMU). Pittsburgh, PA.
T.1
(PDF)
Steve Oney. (2008) Natural Language Search of Structured Documents. M.eng Thesis (MIT). Cambridge, MA, USA.

Grants

10/2020
$60,000
Designing and Building Collaborative Tools for Mixed-Ability Programming Teams
Steve Oney (+ Mauli Pandey, unofficially)
Google Inc.
Award for Inclusion Research Award
09/2020
$499,209
Improving Web Accessibility Through Multi-Resolution Mixed-Initiative Interaction Tools
Steve Oney and Walter Lasecki
National Science Foundation (NSF)
IIS: Cyber-Human Systems (CHS)
07/2019
$10,875
Applying Literate Programming Approaches to Support Semantic Annotation
Andrea Thomer and Steve Oney
The U-M Office of Research (UMOR)
10/2019
$598,926
Scalable Remote Peer Help for Programming Education
Steve Oney, Paul Resnick, and Christopher Brooks
National Science Foundation (NSF)
Improving Undergraduate STEM Education: Education and Human Resources (IUSE: EHR)
01/2018
$174,981
Designing Scalable Help Tools for Programming Courses
Steve Oney
National Science Foundation (NSF)
Cyber-Human Systems (CHS) CRII
11/2017
$37,000
Prototyping Tools to Improve Crowd Based Training for IVA Development
Steve Oney and Walter Lasecki
Clinc, Inc.
04/2017
$198,327
End-User Techniques for Aggregating and Analyzing Exercise and Physical Data
Steve Oney, Michael Nebeling, and Sun Young Park
Michigan Exercise Science & Sports Initiative

Awards

Note: Does not include best paper awards or nominations (in Publications above)
09/2015
University of Michigan President's Postdoctoral Fellowship
10/2009
UIST Student Innovation Contest, 1st place
Part of winning team in most creative category
09/2009
Google/UNCF Scholarship
One-year scholarship for $10,000
09/2009 – 05/2012
Ford Foundation Predoctoral Fellowship
Annual stipend of $20,000 for three years, awarded to 60 doctoral students nationwide across disciplines
09/2008 – 05/2011
ARCS Foundation Scholarship (Pittsburgh Chapter)
Annual stipend of $5,000 for three years. Awarded to 13 doctoral students in the Pittsburgh area (CMU & University of Pittsburgh)
09/2008
MIT Batttlecode Open Programming Competition Finalist
09/2008
NEWMAC Academic All-Conference
Awarded for academic success while a member of MIT's varsity track team

Invited Presentations

10/2020
Cambridge, MA
(remote)
MIT CSAIL HCI Seminar
Designing Tools for Remote Communication in Programming
11/2019
Bloomington, IN
Indiana University
Designing Tools for Remote Communication and Collaboration
11/2018
Williamstown, MA
Williams College
CS Colloquium—Designing Tools for More Effective Remote Communication
11/2017
Madison, WI
University of Wisconsin
HCI Seminar: Designing Tools for Remote Communication Between Programmers
03/2016
Ann Arbor, MI
University of Michigan School of Information
Programming Tools that Speak our Language
10/2015
South Bend, IN
University of Notre Dame Department of Computer Science and Engineering
Expressing Interactivity with States and Constraints
04/2015
Chicago, IL
University of Illinois at Chicago Department of Computer Science
Expressing Interactivity with States and Constraints
03/2015
Boston, MA
Boston University Department of Computer Science
Expressing Interactivity with States and Constraints
03/2015
Palo Alto, CA
FX Palo Alto Laboratory
Expressing Interactivity with States and Constraints
03/2015
Stony Brook, CA
Stony Brook Computer Science Department
Expressing Interactivity with States and Constraints
02/2015
Irvine, CA
University of California at Irvine
Expressing Interactivity with States and Constraints
03/2010
Dagstuhl, Germany
Dagstuhl: Practical Software Testing: Tool Automation and Human Factors
06/2009
San Jose, CA
IBM Almaden Lunch Seminar
FireCrystal: Understanding Interactive Behaviors in Dynamic Web Pages

Service

2021
Program Co-Chair (with Jácome Miguel Costa Cunha)
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)

Organizing Committee

2020
Graduate Consortium co-chair (with Michael Lee)
IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)

Program Committee

2019
International Workshop on Eye Movements in Programming (EMIP)
2019
ACM Conference on Tangible, Embedded, and Embodied Interactions (TEI)
2019
Tech Notes for the ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS)
2017, 2018, 2020
ACM Symposium on User Interface Software and Technology (UIST)
2018, 2020
ACM International Conference on Supporting Group Work (GROUP)
2017 – 2020
Programming Experience Workshop (PX)
2016 – 2020
ACM Conference on Human Factors in Computing Systems (CHI)

Peer Reviewing

2018
IEEE Transactions of Software Engineering (TSE)
2018
ACM Transactions on Computer-Human Interaction (TOCHI)
2010 – 2016
ACM Conference on Human Factors in Computing Systems (CHI)
2011 – 2015
ACM Symposium on User Interface Software & Technology (UIST)
2014
Conference on Human-Computer Interaction with Mobile Devices (MobileHCI)
2010, 2012
ACM Conference on Designing Interactive Systems (DIS)
2008, 2009
Philippine Journal of Science (PJS)

Operations Committee

2012 – 2015
ACM CHI Operations Committee (mobile program guide)
2013 – 2015
ACM UIST Operations Committee (mobile program guide)
2013 – 2015
ACM ITS Operations Committee (mobile program guide)
2012
ACM UbiComp Committee (mobile program guide)
2010, 2011
ACM CHI student volunteer (mobile guide development team)

UMSI

2019 – 2020
School of Information Technical Curriculum Task Force
2018 – 2020
School of Information BSI Committee
2018 – 2019
University of Michigan Interactive and Social Computing (MISC) Coordinator

Other

2009 – 2015
CMU Computer Science outreach roadshow volunteer (with Women@SCS & SCS4All)
2011 – 2014
CMU Human-Computer Interaction Institute (HCII) ombudsman
2010
CMU Human-Computer Interaction Institute (HCII) visit weekend co-chair
2009, 2010
CMU Human-Computer Interaction Institute (HCII) PhD lunch coordinator

Teaching

2021 – present
University of Michigan
Instructor (and creator) – SI 579 (Building Interactive Applications)
2016 – present
University of Michigan
Instructor – SI 106 (Programs, Information, & People)
2009
University of Michigan & Coursera
Python 3 Programming Specialization
https://www.coursera.org/specializations/python-3-programming
Fall 2012
Carnegie Mellon
Instructor – Web Lab, Programming User Interfaces
Developed syllabus, wrote lectures, created projects, presented, graded, and held office hours weekly. Instructor rating: 4.7/5.0
Fall 2010
Carnegie Mellon
Instructor – GUI Lab, Programming User Interfaces
Developed syllabus, wrote lectures, created projects, presented, graded, and held office hours weekly. Instructor rating: 4.6/5.0
Fall 2007 & Spring 2008
MIT
Instructor – GUI Lab, Programming User Interfaces
Taught three recitation sections per week, held weekly office hours, and graded students’ exams
Summer 2007
MIT
Teaching Assistant – Interphase Physics I
Taught three classes per week, held weekly office hours, and mentored a group of incoming MIT freshmen
Spring 2005
MIT
Teaching Assistant – Technology Enabled Learning (TEAL) Physics II
Spring 2005 – Fall 2006
MIT
Laboratory Assistant – Circuits and Electronics
Fall 2006
MIT
Laboratory Assistant – Computational Structures

Students Supervised

Ph.D. Advisees

Fall 2018 – present
University of Michigan
Lei Zhang (School of Information)
(ongoing)
Fall 2018 – present
University of Michigan
(Mauli) Maulishree Pandey (School of Information)
(ongoing, co-advised with Sile O'Modhrain)
Fall 2018 – present
University of Michigan
(April) Yi Wang (School of Information)
(ongoing, co-advised with Christopher Brooks)
Fall 2017 – present
University of Michigan
Rebecca Krosnick (Computer Science and Engineering)
(ongoing)
Fall 2015 – Fall 2020
University of Michigan
Yan Chen (School of Information)
On-Demand Collaboration in Programming

Thesis Committees

2020
University of Michigan
Ph.D.: Vaspol Ruamviboonsuk (Computer Science and Engineering)
Understanding and Improving the Performance of Web Page Loads
2019
University of Michigan
Ph.D.: Shih-Chieh Lin (Computer Science and Engineering)
Cross-Layer System Design for Autonomous Driving
2017
University of Michigan
Ph.D.: Sang Won Lee (Computer Science and Engineering)
Improving User Involvement Through Live, Collaborative Creation
2017
University of Michigan
Ph.D.: Xin Rong (School of Information)
Neural Language Models for Data-Driven Programming Support
2020
University of Michigan
M.S.: (Andy) Xunan Zhou (School of Information)
Conversational Agent Experience: What makes a good skill?
2020
University of Michigan
M.S.: Kangning Chen (School of Information)
Providing Examples and Tool Support for Novice AR Creators
2019
University of Michigan
M.S.: Katy Madier (School of Information)
Enabling Low-cost Co-located Virtual Reality Experiences
2018
University of Michigan
M.S.: Maulishree Pandey (School of Information)
Exploring and Designing for the Self-Tracking Needs of Recreational Athletes

Press

VentureBeat, 2014
Adobe and CMU researchers unveil a brilliant new JavaScript library: ConstraintJS
June 23
Wired, 2013
Researchers Figure Out How You Can Type on a Smartwatch
May 1
Slashdot, 2013
CMU Offers Wee QWERTY Texting Tech for Impossibly Tiny Devices
May 1
Gizmodo, 2013
How Typing on a Smart Watch Might Actually Make Sense
April 29
MIT Tech Review, 2013
A QWERTY Keyboard for your Wrist
April 27

Patent

11/2016
US Patent number 9,495,134. "Methods and Apparatus for Code Segment Handling" Brandt, J. & Oney, S.