---
layout: presentation
title: Interaction Technique Design --Week 7, Wednesday--
description: Description of Interaction Technique Design
class: middle, center, inverse
---
name: inverse
layout: true
class: center, middle, inverse
---
layout:false
# Why is selection (and testing) important .red[*]
.left-column50[
]
.right-column50[
]
.footnote[.red[*] [Washington Post Article](https://www.washingtonpost.com/news/morning-mix/wp/2018/01/16/that-was-no-wrong-button-in-hawaii-take-a-look/?utm_term=.1848969db923)
]
---
class: center, middle, inverse
# Interaction Technique Design
Jennifer Mankoff
CSE 340 Winter 2020
---
layout: false
[//]: # (Outline Slide)
# Today's goals
- Running example: [Pet gallery](https://gitlab.cs.washington.edu/cse340-20wi/cse340-petgallery)
- Introducing key concepts for menus assignment
- Using key properties of people to predict design outcomes
---
# Revisiting event records
Event record:
- Where
- Where is the location of the cursor (x,y position on the screen)
- For focus based events, where is often the target (Lauren's lecture covered this)
- Value is anything you might need to know (like whether it's a 1 or 2 finger swipe; or which key was pressed)
Why is this important? For *picking*:
- When input is positional, *only* interactors whose bounding box includes (x,y) -- where -- are picked
- When input is focus based, *only* interactors in the focus list are picked. Where isn't used in dispatch
---
# Menus Assignment
Will compare pie menus to linear menus
We provide support for running the experiment in MainActivity (and a testing harness in TestActivity)
You implement a variety of menus
---
# Menus Assignment
<div class="mermaid">
classDiagram
class MenuExperimentView {
onTouchEvent()
startSelection()
endSelection()
updateModel()
onDraw()
}
AbstractMenuExperimentView <|-- MenuExperimentView
AbstractMainActivity <|-- MainActivity
AbstractMainActivity <|-- TestActivity
MenuExperimentView <|-- PieMenuView
MenuExperimentView <|-- NormalMenuView
MenuExperimentView <|-- CustomMenuView
</div>
---
# What is a Menu in theory?
???
- supports selection of an item from a fixed set – usually set determined in advance
- typically used for “commands”
- occasionally for selecting a value (e.g., picking a font)
--
.left-column-half[
- supports selection of an item from a fixed set – usually set determined in advance
- typically used for “commands”
- occasionally for selecting a value <br> (e.g., picking a font)
]
--
<div class="mermaid">
graph TD
S((.)) --> A(Start)
A -- "Press?startSelection()" --> I(Selecting)
I -- "Release:endSelection()" --> E[End]
I -- "Drag:updateModel()" --> I
classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px;
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px;
classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF
class S invisible
class A start
class E finish
class I normal
</div>
???
Implemented in MenuExperimentView
Works in all Menus!
---
# What is a Menu in theory?
.left-column-half[
- supports selection of an item from a fixed set – usually set determined in advance
- typically used for “commands”
- occasionally for selecting a value <br> (e.g., picking a font)
Implemented in MenuExperimentView
Works in all Menus!
]
<div class="mermaid">
graph TD
S((.)) --> A(Start)
A -- "Press?startSelection()" --> I(Selecting)
I -- "Release:endSelection()" --> E[End]
I -- "Drag:updateModel()" --> I
classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px;
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px;
classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF
class S invisible
class A start
class E finish
class I normal
</div>
---
# Aside: Enums
.left-column-half[
Group of named constants
- Used for PPS in colorPicker (PPS States; Essential Geometry)
- Used for PPS in Menu assignment *and* for experimental conditions
- Easy to inspect if you want to (can use to determine possible states, number items, etc)
[Documentation](https://docs.oracle.com/javase/tutorial/java/javaOO/enum.html)
]
.right-column-half[
<div class="mermaid">
classDiagram
class TaskType {
LINEAR
RELATIVE
UNCLASS
}
class State {
START
SELECTING
}
class MenuType {
NORMAL
PIE
CUSTOM
}
</div>
]
---
# How do we draw a menu?
.left-column-half[
Coordinates depend on which class you're in
MenuExperimentView set to `MATCH_PARENT`
Makes drawing tricky
Translate (like a parent view does for kids)
]
.right-column-half[
<div class="mermaid">
classDiagram
MenuExperimentView <|.. PieMenuView : onDraw translates and calls drawMenu
class PieMenuView {
Canvas: 0,0 at first touch point.
Clipping rect: Whole screen
drawMenu()
}
class MenuExperimentView {
Canvas : 0,0 at top left of screen;
Clipping rect: Whole screen
onDraw() translate
}
</div>
]
???
Do we have to translate back?
--
We don't have to rotate or translate back because this is
an inheritance drawing trick, not an interactor hierarchy drawing implementation
---
# Aside: Custom Listeners
- Used in ColorPicker and Menus. You'll use them lots
- Why create custom listeners?
--
- Let you execute code when your view's model has changed
- No other way to know that has happened
---
# How to implement
Custom View needs
- To define the custom interface
- To keep track of listeners
--
Anything using the view needs
- To implement the interface (specifically, the method that will be called)
- To register itself as a listener
---
# Example Custom View -- ColorPicker: We setup the Custom View side for you
```java
// Currently registered ColorListener instance or null.
private List<ColorChangeListener> mColorChangeListeners;
// Class which defines a listener to be called when a new color is selected.
public interface ColorChangeListener {
void onColorSelected(@ColorInt int color);
}
// Registers a new listener
public final void addColorChangeListener(@NonNull ColorChangeListener colorChangeListener) {
mColorChangeListeners.add(colorChangeListener);
}
```
---
# Example Custom Listener -- ColorPicker: We implemented this
.left-column-half[
`// TODO: Register callback to update {color,label} View when color changed.`
What method do we call to register the callback? `addColorChangeListener()`
What do we usually do in a callback? update application (`MainActivity`) model
]
.right-column-half[
<div class="mermaid">
classDiagram
MainActivity ..> ColorPickerView : addColorChangeListener()
MainActivity --> ColorChangeListener : Contains
class ColorChangeListener {
onColorChanged()
}
class ColorPickerView {
addColorChangeListener()
}
class MainActivity {
ColorPickerView view
}
</div>
]
---
# Example Custom Listener -- ColorPicker: We implemented this
.left-column-half[
`// TODO: Register callback to update {color,label} View when color changed.`
What method do we call to register the callback? `addColorChangeListener()`
What do we usually do in a callback? update application (`MainActivity`) model
]
.right-column-half[
<div class="mermaid">
classDiagram
ColorPickerView ..> ColorChangeListener : onColorChanged()
ColorChangeListener ..> MainActivity : update model
class ColorChangeListener {
onColorChanged()
}
class MainActivity {
ColorPickerView view
}
</div>
]
---
# You need to do this yourself in Menus
```java
// TODO: register a new listener with the menu so that the application knows when a selection is made
// TODO: implement the listener. When the user completes a trial, the menu listener should store
// the results of the trial, and setup for the next trial
```
---
class: center, middle, inverse
# Experimenting with Interfaces
Important part of buliding interfaces is experimenting
Need structured ways to decide what's better
---
# Experiments should be tied to *hypotheses* based on *theories* what will go better
- Fitts Law (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
- Does someone have to 'check' something? More than once?
- Do they have to move? More than once
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)
???
why theory and not intuition?
---
# Which is better and which laws explain it?
.left-column50[
## A Pie Menu
]
.right-column50[
## B Pull down Menu
]
???
What analysis methods can we use to predict?
- *Fitts Law* (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
- Does someone have to 'check' something? More than once?
- Do they have to move? More than once
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)
---
# Steering Law (based on Fitts' law)
.left-column50[
## A Pie Menu
]
.right-column50[
## B Pull down Menu
]
---
# Which is better and which law explains it?
.left-column50[
## A Pie Menu
]
.right-column50[
## B Marking Menu
![:youtube Video assigned before class, 8c58bN6ajJ4?t=30]
]
???
- Fitts Law (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
- **Does someone have to 'check' something? More than once?**
- **Do they have to move? More than once**
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)
---
# Cognitive modeling (less double checking)
.left-column50[
## A Pie Menu
]
.right-column50[
## B Marking Menu
![:youtube Video assigned before class, 8c58bN6ajJ4?t=30]
]
---
# Which is better and which laws explain it?
A: Tapping B: Crossing
![:youtube Video of using crossing for selection, kp2Zl4ONuik]
???
- **Fitts Law (compare distance and size; check for infinite size)**
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
- Does someone have to 'check' something? More than once?
- Do they have to move? More than once
- Gestalt Psychology (will they see it at all?)
- Errors (will they be reduced)
---
# Fitts' Law (infinite width)
A: Tapping B: Crossing
![:youtube Video of using crossing for selection, kp2Zl4ONuik]
---
# Which is better and which laws explain it?
.left-column50[
## A
]
.right-column50[
## B
]
???
- Fitts Law (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
- Does someone have to 'check' something? More than once?
- Do they have to move? More than once
- Gestalt Psychology (will they see it at all?)
- **Errors (will they be reduced)**
---
# Errors (better physical feedback during motor task)
.left-column50[
## A
]
.right-column50[
## B
]
---
# Which is better and which laws explain it?
.left-column50[
## A
]
.right-column50[
## B
]
???
- Fitts Law (compare distance and size; check for infinite size)
- Steering Law (distance and size over a path)
- Cognitive modeling (complex multi-step model of expert behavior)
- Does someone have to 'check' something? More than once?
- Do they have to move? More than once
- **Gestalt Psychology (group?)**
- **Errors (will they be reduced)**
---
# Gestalt Psychology (better grouping strategies)
.left-column50[
## A
]
.right-column50[
## B
]
---
# Back to comparing menus
Kurtenbach:
![:youtube Illustration of advantages of marking menus,dtH9GdFSQaw]
---
# How do we prove our theories? Hypothesis
<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px;
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px;
classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF
linkStyle 0 stroke-width:4px;
class S invisible
class Hypothesis start
</div>
Discuss
--
Marking Menus are better than Pie Menus
--
Marking Menus are *faster* than Pie Menus
--
Pie Menus are faster than Linear Menus for less than 8 items that have no obvious order
---
.left-column-half[
# What conditions does this experiment have?
`MenuType`
`TaskType` (could be a condition)
]
.right-column-half[
# What might we measure? (from Friday)
]
--
.right-column-half[
- Time on Task
- Accuracy
- How strenuous
- Recall
- Emotional Response
]
<!-- --- -->
<!-- # Classes that handle experimentation -->
<!-- We provide most of the implementation for this -->
<!-- <div class="mermaid"> -->
<!-- classDiagram -->
<!-- class MainActivity { -->
<!-- showMenuForTrial() -->
<!-- } -->
<!-- class AbstractMainActivity { -->
<!-- clearCSV() -->
<!-- } -->
<!-- class MainActivity { -->
<!-- showMenuForTrial() -->
<!-- } -->
<!-- class TestActivity { -->
<!-- showMenu() -->
<!-- } -->
<!-- class TrialListener { -->
<!-- onTrialCompleted() -->
<!-- } -->
<!-- AbstractMainActivity <|-- MainActivity -->
<!-- AbstractMainActivity <|-- TestActivity -->
<!-- class ExperimentSession { -->
<!-- createTrials() -->
<!-- getNext() -->
<!-- next() -->
<!-- } -->
<!-- </div> -->
---
# How do we prove our theories? Method
<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px;
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px;
classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF
linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
class S invisible
class Hypothesis start
class Method normal
</div>
Compare specific conditions
| MenuType | 8 items | 12 items |
|----------|---------|----------|
| Pie | | |
| Marking | | |
| Linear | | |
What if we wanted to do length AND ordered vs not?
Want to consider every combination, or run twice (menuType x length and menuType x ordering)
---
# How do we prove our theories? Data
<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)
classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px;
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px;
classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF
linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
class S invisible
class Hypothesis start
class Method,Data normal
</div>
What might we want to measure? (discussed friday)
--
- Time on task -- how long to complete basic tasks? (For example, find something to buy, create a new account, and order the item.)
--
- Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?)
--
- How strenuous (e.g. gaze has lower throughput but is less strenuous than head pointing (Mackenzie 2018)
--
- Recall -- How much does the person remember afterwards or after periods of non-use?
--
- Emotional Response -- How does the person feel about the tasks completed? (Confident? Stressed? Would the user recommend this system to a friend?)
???
Build up a list of good UI design principals from these basics
Undo
Predictability
... What is missing? (e.g. fun)
---
# How to get such data?
- Video
- Timestamps
- Notes
- Can transcribe and analyze interviews with users
- Can look for patterns across users
---
# How do we prove our theories? Analysis
<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)
Data -- "Clean and Prep" --> Analysis(Analysis)
classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px;
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px;
classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF
linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;
class S invisible
class Hypothesis start
class Method,Data,Analysis normal
</div>
---
# How do we prove our theories? Conclusions
<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)
Data -- "Clean and Prep" --> Analysis(Analysis)
Analysis --> Conclusions(Conclusions)
classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px;
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px;
classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF
linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;
linkStyle 4 stroke-width:4px;
class S invisible
class Hypothesis,Conclusions start
class Method,Data,Analysis normal
</div>
1 = no; 5 = yes
40 responses
---
# How do we prove our theories? Conclusions
<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)
Data -- "Clean and Prep" --> Analysis(Analysis)
Analysis --> Conclusions(Conclusions)
classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px;
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px;
classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF
linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;
linkStyle 4 stroke-width:4px;
class S invisible
class Hypothesis,Conclusions start
class Method,Data,Analysis normal
</div>
---
# How do we prove our theories? Conclusions
<div class="mermaid">
graph LR
S((.)) --> Hypothesis(Hypothesis)
Hypothesis -- "Study Design" --> Method(Method)
Method -- "Run Study" --> Data(Data)
Data -- "Clean and Prep" --> Analysis(Analysis)
Analysis --> Conclusions(Conclusions)
classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px;
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px;
classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF
linkStyle 0 stroke-width:4px;
linkStyle 1 stroke-width:4px;
linkStyle 2 stroke-width:4px;
linkStyle 3 stroke-width:4px;
linkStyle 4 stroke-width:4px;
class S invisible
class Hypothesis,Conclusions start
class Method,Data,Analysis normal
</div>
1 = no; 5 = very much
---
# Further results of class experiment
- 86.7% of students in the class participated in the experiment
- There were no big differences between the two groups
- The first day (Thu 1/9), very few no-dot students brought their own cards. However, starting with the next class, students were pretty good about bringing their own cards
- Students do not seem to bring their cards on Thursday (quiz sections)
- Most students have no trouble in remembering name tags
- Most students used name tents at least 4 times
- Students generally feel okay about using name tents (no strong preference)
- There was no evidence that name tents helped the TA remember students' names
Some factors that may have impacted the data:
- Some students did not attend classes/quiz sections
- Some students with dots attended classes but did not find/pick up their cards
Charts: https://tinyurl.com/wwaw5la
---
<!-- .left-column50[ -->
<!-- ## A -->
<!--  -->
<!-- ] -->
<!-- .right-column50[ -->
<!-- ## B -->
<!--  -->
<!-- ] -->
<!-- ??? -->
<!-- Hypothesis: Errors (will they be reduced) -->
<!-- Method:?? -->
<!-- Data:?? -->
<!-- Analysis?? -->
<!-- Conclusions?? -->
<!-- --- -->
<!-- # How do we prove our theories? Conclusions -->
<!-- <div class="mermaid"> -->
<!-- graph LR -->
<!-- S((.)) -\-> Hypothesis(Hypothesis) -->
<!-- Hypothesis -- "Study Design" -\-> Method(Method) -->
<!-- Method -- "Run Study" -\-> Data(Data) -->
<!-- Data -- "Clean and Prep" -\-> Analysis(Analysis) -->
<!-- Analysis -\-> Conclusions(Conclusions) -->
<!-- classDef finish outline-style:double,fill:#d1e0e0,stroke:#333,stroke-width:2px; -->
<!-- classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px; -->
<!-- classDef start fill:#d1e0e0,stroke:#333,stroke-width:4px; -->
<!-- classDef invisible fill:#FFFFFF,stroke:#FFFFFF,color:#FFFFFF -->
<!-- linkStyle 0 stroke-width:4px; -->
<!-- linkStyle 1 stroke-width:4px; -->
<!-- linkStyle 2 stroke-width:4px; -->
<!-- linkStyle 3 stroke-width:4px; -->
<!-- linkStyle 4 stroke-width:4px; -->
<!-- class S invisible -->
<!-- class Hypothesis,Conclusions start -->
<!-- class Method,Data,Analysis normal -->
<!-- </div> -->
<!-- .left-column50[ -->
<!-- ## A Pie Menu -->
<!--  -->
<!-- ] -->
<!-- .right-column50[ -->
<!-- ## B Pull down Menu -->
<!--  -->
<!-- ] -->
<!-- ??? -->
<!-- Hypothesis: Errors will be reduced; -->
<!-- Hypothesis: Motion will be faster due to low level motor things -->
<!-- Hypothesis: fewer cognitive checks needed -->
<!-- Method:?? -->
<!-- Data:?? -->
<!-- Analysis?? -->
<!-- Conclusions?? -->