ABOUT ME

Anna Maria Feit
NEWS: My dissertation is published and available here. It is a must read for everyone interested in text entry and the optimization of user interfaces.



I am a postdoctoral researcher in Human-Computer Interaction in the AIT Group of Prof. Otmar Hilliges at ETH Zurich.

I received a Doctoral degree from Aalto University in Helsinki where I worked with Prof. Antti Oulasvirta in the User Interfaces group. Before that, I studied Computer Science at Saarland University, Germany.

My research is in the area of computational interaction. I use methods of optimization, machine learning, or mathematical modeling to improve and augment our interaction with computers. Among other forms of input, I am particularly interested in text input methods and have conducted several studies to understand how people type, in small lab settings and through large-scale online typing tests.

I have applied this approach in several projects, including performance modeling and optimization of mid-air gestures, and the transfer of musical expertise to text entry. During an internship at Microsoft Research in Redmond, I worked with the MSR Enable Group and Meredith Ringel Morris on characterizing the accuracy and precision of eye tracking during practical use and implications for gaze-enabled applications.


Address

ETH Zurich, Department of Computer Science,
Office CNB, E 108.2
8092 Zürich
Universitätstrasse 6
Switzerland

Email

feitanna@gmail.com

“A keyboard is for typing words. A piano is for typing emotions.” by @drayzee via twitter

PROJECTS

136M keystrokes

136M Keystrokes

See more
How we type

How we type

See more
Eye tracking quality

Eye tracking quality

See more
Mid-Air Text Entry

Mid-Air Text Entry

See more
PianoText

PianoText

See more

Datasets

"How We Type" Dataset

We released the data collected during the How We Type study. The dataset is very large. It contains the motion capture data (movement of hands and fingers during typing), the keystrokes, reference videos, and eye tracking data. The smaller parts of the dataset can be directly downloaded on the project page. Please contact me if you want other parts.

"136M Keystrokes" Dataset

We collected typing data of over 160,000 users participating in an online typing test. This dataset is available on the project page. It contains the keypress and -release events with details of each keystroke.

Code and Resources

Automatic Labeling of Motion Capture Markers (for Hand tracking)

As part of the How We Type project I developed Python scripts to automatically label motion capture markers based on a nearest neighbor approach. The scripts are free to use and can be found on GitHub


PUBLICATIONS AND TALKS

Dissertation

Assignment Problems for Optimizing Text Input
Aalto University publication series DOCTORAL DISSERTATIONS, 103/2018

Annas dissertation

My PhD thesis advances the state of the art in text-entry optimization by proposing novel objective functions that quantify the performance, ergonomics and learnability of a text input method. In addition, the thesis gives an introduction to user interface optimization and summarizes work in text entry optimization from the last 80 years. The assignment problem thereby serves as a unifying framework for formulating the novel objectives as well as those from prior work. I show the optimization of text input in three concrete cases: text input with multi-finger gestures in mid-air, text input on a long piano keyboard, and -- for a contribution to the official French keyboard standard -- input of special characters via a physical keyboard. While the work focused on text input, the assignment problem can be used to model other design problems in HCI (e.g., how best to assign commands to UI controls or distribute UI elements across several devices), for which the same problem formulations, optimization techniques, and even models could be applied.

Full papers

Honorable Mention AwardObservations on Typing from 136 Million Keystrokes
Conference on Human Factors in Computing Systems, CHI 2018. Honourable Mention Award.
PDF, 2.8MB BibTeX Project page Slides

Vivek Dhakal, Anna Maria Feit, Per Ola Kristensson, Antti Oulasvirta

We report on typing behaviour and performance of 168,000 volunteers in an online study. The large dataset allows detailed statistical analyses of keystroking patterns, linking them to typing performance. Besides reporting distributions and confirming some earlier findings, we report two new findings. First, letter pairs that are typed by different hands or fingers are more predictive of typing speed than, for example, letter repetitions. Second, rollover-typing, wherein the next key is pressed before the previous one is released, is sur- prisingly prevalent. Notwithstanding considerable variation in typing patterns, unsupervised clustering using normalised inter-key intervals reveals that most users can be divided into eight groups of typists that differ in performance, accuracy, hand and finger usage, and rollover. The code and dataset are released for scientific use.

AdaM: Adapting Multi-User Interfaces for Collaborative Environments in Real-Time
Conference on Human Factors in Computing Systems, CHI 2018.
PDF, 4MB BibTeX Project page

Seonwook Park, Christoph Gebhardt, Roman Rädle, Anna Maria Feit, Hana Vrzakova, Niraj Dayama, Hui-Shyong Yeo, Clemens Klokmose, Aaron Quigley, Antti Oulasvirta, Otmar Hilliges

Developing cross-device multi-user interfaces (UIs) is a challenging problem. There are numerous ways in which content and interactivity can be distributed. However, good solutions must consider multiple users, their roles, their preferences and access rights, as well as device capabilities. Manual and rule-based solutions are tedious to create and do not scale to larger problems nor do they adapt to dynamic changes, such as users leaving or joining an activity. In this paper, we cast the problem of UI distribution as an assignment problem and propose to solve it using combinatorial optimization. We present a mixed integer programming formulation which allows real-time applications in dynamically changing collaborative settings. It optimizes the allocation of UI elements based on device capabilities, user roles, preferences, and access rights. We present a proof-of-concept designer-in-the-loop tool, allowing for quick solution exploration. Finally, we compare our approach to traditional paper prototyping in a lab study.

Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands
Conference on Human Factors in Computing Systems, CHI 2018.
PDF, 8.2MB BibTeX

Pascal Knierim, Valentin Schwind, Anna Maria Feit, Florian Nieuwenhuizen, Niels Henze

Entering text is one of the most common tasks when interacting with computing systems. Virtual Reality (VR) presents a challenge as neither the user's hands nor the physical input devices are directly visible. Hence, conventional desktop peripherals are very slow, imprecise, and cumbersome. We developed a apparatus that tracks the user's hands, and a physical keyboard, and visualize them in VR. In a text input study with 32 participants, we investigated the achievable text entry speed and the effect of hand representations and transparency on typing performance, workload, and presence. With our {apparatus}, experienced typists benefited from seeing their hands, and reach almost outside-VR performance. Inexperienced typists profited from semi-transparent hands, which enabled them to type just 5.6 WPM slower than with a regular desktop setup. We conclude that optimizing the visualization of hands in VR is important, especially for inexperienced typists, to enable a high typing performance.

Selection-based Text Entry in Virtual Reality
Conference on Human Factors in Computing Systems, CHI 2018.
PDF, 2.5MB BibTeX

Marco Speicher, Anna Maria Feit, Pascal Ziegler, Antonio Krüger

In recent years, Virtual Reality (VR) and 3D User Interfaces (3DUI) have seen a drastic increase in popularity, especially in terms of consumer-ready hardware and software. While the technology for input as well as output devices is market ready, only a few solutions for text input exist, and empirical knowledge about performance and user preferences is lacking. In this paper, we study text entry in VR by selecting characters on a virtual keyboard. We discuss the design space for assessing selection-based text entry in VR. Then, we implement six methods that span different parts of the design space and evaluate their performance and user preferences. Our results show that pointing using tracked hand-held controllers outperforms all other methods. Other methods such as head pointing can be viable alternatives depending on available resources. We summarize our findings by formulating guidelines for choosing optimal virtual keyboard text entry methods in VR.

Computational Support for Functionality Selection in Interaction Design
Journal Paper, ACM TOCHI, 2017
ACM DL Author-ize service PDF, 3.2MB BibTeX Project page Slides

Antti Oulasvirta, Anna Maria Feit, Perttu Lähteenlahti, Andreas Karrenbauer

Designing interactive technology entails several objectives, one of which is identifying and selecting appropriate functionality. Given candidate functionalities such as “print,” “bookmark,” and “share,” a designer has to choose which functionalities to include and which to leave out. Such choices critically affect the acceptability, productivity, usability, and experience of the design. However, designers may overlook reasonable designs because there is an exponential number of functionality sets and multiple factors to consider. This article is the first to formally define this problem and propose an algorithmic method to support designers to explore alternative functionality sets in early stage design. Based on interviews of professional designers, we mathematically define the task of identifying functionality sets that strike the best balance among four objectives: usefulness, satisfaction, ease of use, and profitability. We develop an integer linear programming solution that can efficiently solve very large instances (set size over 1,300) on a regular computer. Further, we build on techniques of robust optimization to search for diverse and surprising functionality designs. Empirical results from a controlled study and field deployment are encouraging. Most designers rated computationally created sets to be of the comparable or superior quality than their own. Designers reported gaining better understanding of available functionalities and the design space.

Honorable Mention AwardToward Everyday Gaze Input: Accuracy and Precision of Eye Tracking and Implications for Design
Conference on Human Factors in Computing Systems, CHI 2017. Honourable Mention Award.
ACM DL Author-ize service PDF, 7.8MB BibTeX Slides

Anna Maria Feit, Shane Williams, Arturo Toledo, Ann Paradiso, Harish Kulkarni, Shaun Kane, and Meredith Ringel Morris

For eye tracking to become a ubiquitous part of our everyday interaction with computers, we first need to understand its limitations outside rigorously controlled labs, and develop robust applications that can be used by a broad range of users and in various environments. Toward this end, we collected eye tracking data from 80 people in a calibration-style task, using two different trackers in two lighting conditions. We found that accuracy and precision can vary between users and targets more than six-fold, and report on differences between lighting, trackers, and screen regions. We show how such data can be used to determine appropriate target sizes and to optimize the parameters of commonly used filters. We conclude with design recommendations and examples how our findings and methodology can inform the design of error- aware adaptive applications.

How We Type: Movement Strategies and Performance in Everyday Typing
Conference on Human Factors in Computing Systems, CHI 2016
ACM DL Author-ize service PDF, 3.51MB BibTeX Slides Project page

Anna Maria Feit, Daryl Weir, and Antti Oulasvirta.

This paper revisits the present understanding of typing, which originates mostly from studies of trained typists using the ten-finger touch typing system. Our goal is to characterise the majority of present-day users who are untrained and employ diverse, self-taught techniques. In a transcription task, we compare self-taught typists and those that took a touch typing course. We report several differences in performance, gaze deployment and movement strategies. The most surprising finding is that self-taught typists can achieve performance levels comparable with touch typists, even when using fewer fingers. Motion capture data exposes 3 predictors of high performance: 1) unambiguous mapping (a letter is consistently pressed by the same finger), 2) active preparation of upcoming keystrokes, and 3) minimal global hand motion. We release an extensive dataset on everyday typing behavior.

Investigating the Dexterity of Multi-Finger Input for Mid-Air Text Entry
Conference on Human Factors in Computing Systems, CHI 2015
ACM DL Author-ize service PDF, 3.51MB BibTeX Project page

Srinath Sridhar, Anna Maria Feit, Christian Theobalt, and Antti Oulasvirta.

This paper investigates an emerging input method enabled by progress in hand tracking: input by free motion of fingers. The method is expressive, potentially fast, and usable across many settings as it does not insist on physical contact or visual feedback. Our goal is to inform the design of high-performance input methods by providing detailed analysis of the performance and anatomical characteristics of finger motion. We conducted an experiment using a commercially available sensor to report on the speed, accuracy, individuation, movement ranges, and individual differences of each finger. Findings show differences of up to 50% in movement times and provide indices quantifying the individuation of single fingers. We apply our findings to text entry by computational optimization of multi-finger gestures in mid-air. To this end, we define a novel objective function that considers performance, anatomical factors, and learnability. First investigations of one optimization case show entry rates of 22 words per minute (WPM). We conclude with a critical discussion of the limitations posed by human factors and performance characteristics of existing markerless hand trackers.

Redesigning the Piano Keyboard for Text Entry
Conference on Designing Interactive Systems, DIS 2014
ACM DL Author-ize service PDF, 5.56 MB BibTeX Project page

Anna Maria Feit and Antti Oulasvirta.

Inspired by the high keying rates of skilled pianists, we study the design of piano keyboards for rapid text entry. We review the qualities of the piano as an input device, observing four design opportunities: 1) chords, 2) redundancy (more keys than letters in English), 3) the transfer of musical skill and 4) optional sound feedback. Although some have been utilized in previous text entry methods, our goal is to exploit all four in a single design. We present PianoText, a computationally designed mapping that assigns letter sequences of English to frequent note transitions of music. It allows fast text entry on any MIDI-enabled keyboard and was evaluated in two transcription typing studies. Both show an achievable rate of over 80 words per minute. This parallels the rates of expert Qwerty typists and doubles that of a previous piano-based design from the 19th century. We also design PianoText-Mini, which allows for comparable performance in a portable form factor. Informed by the studies, we estimate the upper bound of typing performance, draw implications to other text entry methods, and critically discuss outstanding design challenges.

Workshops, Posters, etc.

Computational Design of Input Methods
Doctoral Consortium, CHI 2017

Designing a user interface or input method requires to evaluate and trade-off many criteria. The corresponding design spaces are huge, making it impossible to consider every potential design. Therefore, my work focuses on the use of computational methods for the design of input methods. I follow a modelling-optimization approach: understand and model the characteristics of the interaction, formulate the design space and develop (multi-) objective functions to evaluate designs, and develop algorithms to systematically search for the best design. In my projects I applied this approach to develop better text entry methods. Among others, I modelled the performance and anatomical constraints of the hand to computationally optimize multi-finger gestures for mid-air input, and studied how people type on physical keyboards, in order to understand and model the performance of two-hand typing.

Towards Multi-Objective Optimization for UI Design
Workshop Paper, CHI 2015
PDF, 228KB BibTeX

Anna Maria Feit, Myroslav Bachynskyi and Srinath Sridhar.

In recent years computational optimization has been applied to the problem of finding good designs for user interfaces with huge design spaces. There, designers are struggling to integrate many different objectives into the design process, such as ergonomics, learnability or performance. However, most computationally designed interfaces are optimized with respect to only one objective. In this paper we argue that multi-objective optimization is needed to improve over manual designs. We identify 8 categories that cover design principles from UI design and usability engineering. We propose a multi-objective function in form of a linear combination of these factors and discuss benefits and pitfalls of multi-objective optimization.

Text is in the Air... Investigating Multi-Finger Gestures for Mid-Air Text Entry
ACM womEncourage 2015
PDF, 676KB

Anna Maria Feit, Srinath Sridhar, Christian Theobalt and Antti Oulasvirta.

Mid-air input, enabled by recent progress in computer vision based marker-less hand tracking, is an exciting candidate for text entry where direct touch input is not practicable or not available. In this paper we investigate the use of chord-like multi-finger gestures for entering text in mid-air. In contrast to previous methods, they require no extrinsic targets and can be performed eyes-free. Wesystematically explore the design space of hand gestures by computationally optimizing the letter-to-gesture mapping with respect to multiple objectives: performance, anatomical comfort, learnability and mnemonics. First investigations of one optimization case show entry rates of 22 words per minute. While this is promising, our study reveals several limitations of both, the mapping design as well as the available tracking methods. We discuss open challenges in mid-air input and conclude with recommendations for future work.

Invited Talks

Optimizing Special Character Entry: the Case of the French Keyboard Standard
Int. Conference on Operations Research OR, 2017
Anna Maria Feit, Antti Oulasvirta

We present the optimization of the new French standard for keyboard layouts. The typical French keyboard layout made it overly complicated or even impossible to type all French characters, punctuation marks, and other symbols. The goal of this new standard was to enable and facilitate the correct spelling of French, and allow the entry of symbols common in programming languages, mathematical expressions, and other European languages. Therefore, we had to assign over 115 characters to 148 slots on the keyboard, optimizing their placement with respect to each other and to the numbers (0-9) and non-accentuated letters (a-z) that were kept unchanged. In interaction with an expert committee, we identified four objectives: input performance and ergonomics, similarity to the prior keyboard, and coherent placement of similar symbols. We collected extensive performance data, language statistics, expert ratings on character similarities, and ergonomic scores to implement and solve a multi-objective quadratic assignment problem. In an iterative process, we computed and adapted a range of solutions in interaction with language experts. The resulting design unifies mathematical bounds with expert opinions and constraints. The new French keyboard layout is the first modern standard where computational optimization methods were used in interaction with domain experts to implement an optimal keyboard design.

Optimization of Text Input
Dagstuhl Seminar on Computational Interactivity, 2017

Designing a user interface or input method requires evaluating and trading-off many criteria. The corresponding design spaces are huge, making it impossible to manually build and test every potential design. For example, if we want to design a method to enter letters in mid-air via finger gestures, there are 10^33 possibilities to assign 27 characters to 32 hand gestures. Therefore, my work focuses on using optimization methods for the design of (text input) systems. The use of optimization methods allows us to efficiently and rigorously search very large design spaces, it gives quantitative guarantees on the goodness of a design and helps us to explicitly formulate and trade-off different criteria and constraints. Using optimization methods to design an input method or user interface requires 3 steps. The first step is how to formulate the design problem. This requires to explicitly state the decisions and constraints in a mathematical way and helps to understand the characteristics of the problem. Second, we need to formulate an objective function that can be used to evaluate and compare different designs. The input data and models we use for evaluation determines the outcome of the optimization. Third, in order to solve the optimization problem, we need computational search methods. This can be mathematical solvers which guarantee to find the optimum, or it can be approximation algorithms. I present several projects in which the focus is to develop more plausible, empirically valid formulations and objectives for more realistic optimization approaches. Among others, we modeled the performance and anatomical constraints of the hand to computationally optimize multi-finger gestures for mid-air input [1] and studied how people type on physical keyboards, in order to understand the performance of two-hand typing [2]. Most recently, we used Integer Programming to optimize the special character layout of the French keyboard to facilitate typing of correct French. Therefore, we reformulated the commonly used letter assignment problem to make it applicable to large real-world cases with over 120 to-be-mapped characters and quantified the performance, ergonomics, and ease-of-use of a keyboard layout. While my work focuses on text entry, the same problem formulations and optimization methods can be applied to the design of many other input methods and user interfaces.

EXPERIENCE

Education and Skills
I studied Computer Science at the Saarland University in Saarbrücken, Germany, which ranks among the top universities for Computer Science worldwide. I graduated with an honours degree, being among the top 5%.

I finished my doctoral studies at Aalto University. I passed my dissertation an defense "with distinction", meaning that it was judged to be among the top 10% of dissertations in the field of Human-Computer Interaction. My areas of expertise are now in the optimization of user interfaces, text entry systems, and user modelling. My skills particularly include the software development with Java and Python, and data analysis using Matlab and Python.

During my studies and research projects I gained a strong background across the multiple disciplines of Computer Science, and skills relevant for academic and industrial work, including:
  • Data analysis and modelling
  • Empirical methods for quantitative and qualitative research
  • Integer programming and meta-heuristic optimization
  • User modeling
  • Eye tracking
  • Machine learning
  • Crowdsourcing
  • Music signal processing
  • Computational logic and automated theorem proofing

Education

Ph.D., HCI, Aalto University- 2014-2018

Doctoral studies in the User Interfaces group of Prof. Antti Oulasvirta in Helsinki, Finland.
Pass with Distinction

Dissertation topic: Assignment Problems for Optimizing Text Input.

M.Sc. Computer Science, Saarland University- 2012 - 2013

Master studies in Computer Science at the Saarland University in Saarbrücken, Germany.
GPA: 1.2, Honours Degree (range: 1.0 (best) - 4.0)

Thesis topic: PianoText: Transferring Musical Expertise to Text Entry.

B.Sc. Computer Science, Saarland University- 2008 - 2012

Bachelor studies in Computer Science at the Saarland University in Saarbrücken, Germany. Minor in Computational Linguistics.
GPA: 1.6 (range: 1.0 (best) - 4.0)

Thesis topic: 3D Room Designer: a Collaborative Web Application Based on XML3D and Sirikata.

Research

ETH Zurich- since 2018

Postdoctoral researcher in the AIT group of Prof. Otmar Hilliges at ETH Zurich, Switzerland.

Research topics: Computational interaction, user interface optimization. Teaching and supervision of students.

Aalto University- 2014 - 2018

Doctoral researcher in the User Interfaces group of Prof. Antti Oulasvirta in Helsinki, Finland.

Research topic: optimization of text input methods.

Microsoft Research- Summer 2016

Summer intern at Microsoft Research, Redmond, with the MSR Enable Group and Meredith Ringel Morris

Research topic: Eye tracking, text entry by eye gaze, adaptive gaze-enabled applications.

Max-Planck Institute for Informatics- 2012-2014

Researcher and student assistant in the Human-Computer Interaction group of Prof. Antti Oulasvirta at the Max-Planck Institue for Informatics and the Cluster of Excellence on Multimodal Computing and Interaction (MMCI) in Saarbrücken, Germany.

Research topics included, the transfer of musical expertise to text entry and the optimization of text entry methods.

German Research Center for AI (DFKI)- 2010-2011

Student assistant in the research group for Agents and Simulated Reality by Prof. Philipp Slusallek at the German Research center for artificial intelligence (DFKI), in Saarbrücken, Germany.

Research topics included the development of 3D web applications with XML3D.

CONTACT ME



Email address:
feitanna@gmail.com

Address:
ETH Zurich, Department of Computer Science,
Office CNB, E 108.2
8092 Zürich
Universitätstrasse 6
Switzerland