Updated avery's slides for Monday

0148df5c · Jennifer Mankoff · 554b71de · 0148df5c · 0148df5c
Commit 0148df5c authored 1 year ago by Jennifer Mankoff
--- a/assets/css/my-remark.scss
+++ b/assets/css/my-remark.scss
@@ -607,6 +607,7 @@ html.remark-container, body.remark-container {
    width: 20%;
    float: left;
    height: 92%;
+    padding-top: .5em;    
    h2:last-of-type, h3:last-child {
 	color: $darkest;
 	font-size: $h2-font;
@@ -617,7 +618,6 @@ html.remark-container, body.remark-container {
 .right-column {
    width: 75%;
    float: left;
-    padding-top: 1em;
    padding-left: 1em;
    font-size: $body-font;
 }

--- a/slides/evaluation.html
+++ b/slides/evaluation.html
 ---
 layout: presentation
 title: Evaluation  --Week 4--
-description: Accessibility Needfinding and Evaluation
+description: Accessibility Need finding and Evaluation
 class: middle, center, inverse
 ---
 background-image: url(img/people.png)
@@ -28,7 +28,7 @@ class:
 [//]: # (Outline Slide)

 # Learning Goals for Today
- Presenting Acessibly
+- Presenting Accessibly
 - How to build inclusive experiment designs
 - Running an inclusive summative study

@@ -38,7 +38,7 @@ class:

 So you have an app and you think it's accessible. How do you check that? 

-.quote[Unlike formative evaluations, whose goals is to inform the design process, summative evaluations involve getting the big picture and assessing the overall experience of a finished product...] - Nielsen Norman Group
+.quote[...getting the big picture and assessing the overall experience of a finished product...] - Nielsen Norman Group

 Lots of variables here, but 
 - typically checks that an app works as expected on standard tasks when used by the target population
@@ -55,7 +55,7 @@ Lots of variables here, but

 Of course the entire design process *should not just include majority class people* and much of this presentation applies to formative testing took

-We'll also cover that later today when we talk about designing apps *for* and *with* people with disabilities
+We'll also cover that later this week when we talk about designing apps *for* and *with* people with disabilities

 ---

@@ -71,9 +71,49 @@ Best guidance: [Nielsen-Norman Group](http://www.nngroup.com/reports/accessibili
 ---

 # Accessible Study Planning Workflow
-![:img A flow chart for planning accessible studies. The first two phases occur in parallel and are "identify stakeholders" and "identify tasks". These tasks flow into the "assign tasks" phase where one should consider access needs, effects of familiarity with the task, personal preferences, and other institutional constraints. The next phase is "planning accommodations", where one can consider the dimensions of communication, materials, time and space. And the final phase is reflect on the study plan while considering access synergies and conflicts and power dynamics.](img/wk04/access_workflow.PNG)

-.red[*needs arrows going back]
+<div class="mermaid">
+%%{init: {'theme':'base', 'themeVariables': { 'primaryColor': '#4CAF50', 'tertiaryColor': '#009688', 'fontSize': '16px', 'textMargin': '0px', 'text-align': 'left' }}%%
+
+flowchart LR
+
+    A(Identify stakeholders)
+    B(Identify tasks)
+    
+    A <--> C
+    B <--> C
+    C <--> D
+    D <--> E
+	
+    subgraph Assign tasks
+    C(Access needs
+      Effects of familiarity
+      Personal preferences
+      Institutional constraints)
+    end
+
+    subgraph Plan Accomm.
+      D(Communication 
+             Materials 
+             Time
+             Space)
+    end
+
+    subgraph Reflect
+       E(Access Synergies
+         Access conflicts
+         Power Dynamics)
+    end
+
+
+classDef default fill:#009688,stroke:#333,stroke-width:2px, color:white;
+classDef reflect fill:#4CAF50,stroke:#333,stroke-width:2px, color:white;
+class A,B,C,D default
+class A,B,C,D,E reflect
+linkStyle default stroke: black,stroke-width:8px
+
+</div>
+

 ---

@@ -85,52 +125,55 @@ Supposed you are hired to perform usability testing of new food-delivery app wit

 # Identify Stakeholders

-Supposed you are hired to perform usability testing of new food-delivery app with people with sensory disabilities.
- Who are the stakeholders who you need to take into account for this study?
+.left-column[you are hired to perform usability testing of new food-delivery app with people with sensory disabilities.]
+
+.right-column[
+Who are the stakeholders who you need to take into account for this study?

-  - Disabled participants
-  	- Blind people
-	- Deaf people
-	- Hard-of-hearing people
+  - Disabled participants (BVI, DHH)
  - Research team
  	- Lua -- lead researcher, has dyslexia
  	- Jay -- project team member, Deaf person who uses sign language.
  	- El -- project team member, non-disabled
-
+]
 ---

 # Design Study

-Supposed you are hired to perform usability testing of new food-delivery app with people with sensory disabilities.
+.left-column[you are hired to perform usability testing of new food-delivery app with people with sensory disabilities.]
+.right-column[
 - Who are the stakeholders who you need to take into account for this study?
 - What is the right structure for the study?
-
+]
 ---
-# Design Study (1 of 4)
+# Design Study (1 of 5)

 This is a usability study, so it should include the same tasks as tested for users without disabilities

 Metrics should be similar as well -- for example the [System Usability Scale](https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html) might be useful at the end

 ---
-# Design Study (2 of 4) 
+# Design Study (2 of 5) 

 This is also a study of accessibility, with disabled users. The study design needs to consider 
-.left-column50[
+
 - Choice of method 
  - e.g. think aloud may be more difficult for a nonspeaking individual, or someone with fatigue
  - Check in on study measures and statistical design. Does user heterogeneity impact this?
 <!-- - Study venue (It may be difficult to duplicate participant hardware/software) -->
-]
-.right-column50[
+
+---
+# Design Study (3 of 5) 
+
+This is also a study of accessibility, with disabled users. The study design needs to consider 
+
 - Ensure that participants' hardware/software also addresses study needs (e.g. do participants have a working monitor?)
 - Study length (estimate before and after piloting)
-   - Study length may increase for some disabled participnats
+   - Study length may increase for some disabled participants
   - Study length may negatively impact participants with fatigue-related concerns
-]

 ---
-# Design Study (3 of 4)
+# Design Study (4 of 5)

 Study metrics may also need revision.
 .left-column50[
@@ -141,26 +184,25 @@ Consider adding
 ]
 .left-column50[
 Consider changing
- Any metrics that include abelist assumptions
+- Any metrics that include ableist assumptions
 - Approach to time based measurements
 ]


 ---
-# Design Study (4 of 4) 
+# Design Study (5 of 5) 

 It is also important to consider ethical issues
-.left-column60[
+
 - What responsibility do you have to participants in terms of providing skills and help in & out of study?
 - What responsibility do you have to give participants a participatory role in design?
 - How do you ensure the integrity of your record of their experience?
 - Are you compensating them appropriately and addressing costly transportation needs?
-]
-.right-column40[
+
+???
 These changes may in turn impact
 - Expense 
 - Scalability 
-]

 ---

@@ -181,10 +223,6 @@ To perform standard usability testing on the app based on disabilities represent
 3. Handle recruitment 
 4. Pilot with people with disabilities
 5. Run the Study
-    - Prepare study materials (e.g., print out protocols)
-    - Hire access support staff
-    - Participate in the study
-    - Compensate participants
 6. Analyze the data

 ---
@@ -236,11 +274,19 @@ Some things they might discuss include
  - When to help and when to end a task if participants have trouble
 ---

-# 3. Handle Recruitment
-As a person already connected to the disability community, the team determins that Jay will be most likely to be trusted, and to identify opportunities to build trust throughout the recruitment process
+# 3. Handle Recruitment (1/2)
+As a person already connected to the disability community, the team determines that Jay will be most likely to be trusted, and to identify opportunities to build trust throughout the recruitment process. 

+Recruitment raises a number of access issues
 - Are your recruitment materials accessible
- How will you find participants?
+- How will you find participants (next slide)
+- How will you compensate participants?
+
+---
+
+# 3. Handle Recruitment (2/2)
+
+Finding participants may require careful work
 - How much do you know about the group your are targeting and what to expect from/of them?
 - Will they see you as trustworthy? 
 - Are you giving them enough agency in the study process?
@@ -266,6 +312,7 @@ Prepare:
 - Hire interpreters (at least 2 weeks in advance)
 - Print out accessible study materials, consent form, etc
 - Set up the room
+- Have compensation ready
 ]

 ---
@@ -274,7 +321,7 @@ Prepare:
 - Lua gets tired from reading overtime
 - Jay knows ASL and Deaf norms
 - Study length was problematic for one Blind participant who also had a chronic illness
- Sometimes sightlines made it hard for Jay to see the interpreter
+- Sometimes sight lines made it hard for Jay to see the interpreter

 ---

@@ -303,7 +350,9 @@ Prepare:

 # 5. Run the Study

-At this point the team should be well prepared. However, it is still important to monitor both the team's and the participants' needs and adjust as accessibility concerns arise.
+At this point the team should be well prepared. However, it is still important to monitor both the team's and the participants' needs and adjust as accessibility concerns arise. 
+
+Similar to piloting a study 

 ---

@@ -315,7 +364,7 @@ This is another area where access concerns may come up.
 - <q>Member Checking</q>

 ---
-# Try it 
+# (If time) Try it 

 You read about Alexa's use by people with disabilities in class. Form small groups
 - Identify potential tasks to test