Merge branch 'week4-changes' into 'UACCESS-23fa'

Week4 changes -- updates from jen (v1)

See merge request undergrad-accessibility-website!41

assets/css/style.scss

@@ -146,7 +146,6 @@ dl dt.homework {
     color: $dark-color;
     padding:5px;
     margin-top: 1em !important;
-    margin-left: 1em !important;
 }
 
 dl dt.activity {

assignments/index.md

@@ -46,7 +46,7 @@ averaged over all the times that you are assessed.
 |--------------------------------------------------------|---------------------------------------------------------|
 | Disability Justice                                     | HW2, Final Project                                      |
 | Finding First Person Accounts of Disability Technology | HW1, Final Project                                      |
-| Familiarity with a Range of Accessibility technologies | HW1, HW4a (by choice)                                   |
+| Familiarity with a Range of Accessibility technologies | HW1, HW5 (by choice)                                   |
 | Appropriate And Inclusive Language                     | Any Writing or Speaking about Disability                |
 | Presenting Accessibly                                  | HW1, Final Project                                      |
 
@@ -54,11 +54,11 @@ averaged over all the times that you are assessed.
 
 | Competency                                            | Assessed in Which Assignment?                           |
 |-------------------------------------------------------|---------------------------------------------------------|
-| Accessibility Problem Fixes                           | HW3b, Final Project (by choice)                         |
-| Accessibility Technology Based Accessibility Checking | HW3a, Final Project (required)                          |
-| Automated Accessibility Checking                      | HW3a, Final Project (by choice)                         |
-| Correct use of Web/App Accessibility Rules            | HW3b, Final Project (by choice)                         |
-| Plain Language Writing                                | HW2, HW4a (by choice), Any Reading response (by choice) |
+| Accessibility Problem Fixes                           | HW4, Final Project (by choice)                         |
+| Accessibility Technology Based Accessibility Checking | HW3, Final Project (required)                          |
+| Automated Accessibility Checking                      | HW3, Final Project (by choice)                         |
+| Correct use of Web/App Accessibility Rules            | HW4, Final Project (by choice)                         |
+| Plain Language Writing                                | HW2, HW5 (by choice), Any Reading response (by choice) |
 
 ## Accessible Media
 
@@ -66,9 +66,9 @@ averaged over all the times that you are assessed.
 |---------------------------------|---------------------------------|
 | Accessible Document Creation    | All                             |
 | Accessible Presentation Creation | HW1, Final Project              |
-| Image Description               | HW1, HW3a, Hw4a, Final Project  |
-| Accessible Website/App Creation | HW4b, Final Project (required)  |
-| Accessible Video Creation       | HW4b, Final Project (by choice) |
+| Image Description               | HW1, HW3, Hw5, Final Project  |
+| Accessible Website/App Creation | HW6, Final Project (required)  |
+| Accessible Video Creation       | HW6, Final Project (by choice) |
 
 
 

schedule.md

@@ -159,15 +159,19 @@ Practice with Accessibility Assessment
 {: .week}
 # Week 4 (10/16-10/20): Building & Remediating Accessible Interfaces
 
-{% details Learning Goals %} 
+{% details Class Learning Goals & Plan %} 
 - How accessibility works under the hood
 - More on navigation and its impact 
 - What are the trade offs between different tools?
-{% enddetails %}
 
-{% details Monday: What's the best way to Assess a Website?%} 
+**Monday Slides** {% include slide.html title="How to build for accessibility" loc="best-practices.html" %}
+
+**Wednesday Slides** {% include slide.html title="Comparing Automated Testing and Manual Testing" loc="comparing-approaches.html" %}
 
-**Slides** {% include slide.html title="Comparing Automated Testing and Manual Testing" loc="comparing-approaches.html" %}
+**Thursday Section** Group work on Report
+
+**Friday Slides** {% include slide.html title="What is an Accessibility Tree" loc="tbd.html" %}
+Maybe a hands on exercise. Also discuss Aria
 
 {: .homework} Required Reading and Reflection (for Wednesday)
 : 
@@ -183,34 +187,15 @@ Practice with Accessibility Assessment
    - Watch: [Latte: Use-Case and Assistive-Service Driven Automated Accessibility Testing Framework for Android](https://www.youtube.com/watch?v=Z84jeO0UZFE) and read [Lies, Damned Lies, Overlays, and Widgets](https://www.linkedin.com/pulse/lies-damned-overlays-widgets-timothy-springer/)
    - Read: [Semantics for Eye tracking](https://dl.acm.org/doi/pdf/10.1145/3338844)
 
-{% enddetails %}
-
-{% details Wednesday:  How to Build Accessible Interfaces %} 
-
-**Slides** {% include slide.html title="How to build for accessibility" loc="best-practices.html" %}
-
 {: .homework} [Website Report](assignments/website-report.html) Assigned: Write a group report about your assessment
 : 
 
-{% enddetails %}
- 
-{% details Thursday: Section %} 
-Group work on Report
-{% enddetails %}
-
-{% details Friday: More Website Standards %} 
-
-**Slides** {% include slide.html title="More Accessibility Standards" loc="assessment2.html" %}
-   
-{: .homework} **Post-module Reflection:**  [Accessibility Basics Reflection](https://forms.gle/Wk8C51m5Sbq8RX7W8)
-: 
-
 {% enddetails %}
 
 {: .week}
 # Week 5 (10/23 - 10/27): Accessible Need-finding and Evaluation
 
-{% details Learning Goals %} 
+{% details Class Learning Goals & Plan %} 
 - How to assess whether a technology is accessible; and whether an accessibility technology is useful and usable, in an inclusive fashion.
 - How do you make sure your *product is accessible* to people with disabilities
    1. Data Equity and implicit bias
@@ -220,11 +205,15 @@ Group work on Report
    - What are potential data sources for assessing value
    - Collaboration Versus Paternalism
    - Overly narrow views of disability: Multiple disabled people & multiply disabled people
-{% enddetails %}
 
-{% details Monday 10/23: Introduction to Assessing Accessibility %} 
+**Monday Slides** {% include slide.html title="Guest Lecture: Avery Mack: Assessing Accessibility" loc="evaluation.html" %}
+
+**Wednesday 10/25** Conversation with Jaipreet Virdi %} 
+- Dr. Jaipreet Virdi will be lecturing on 10/24 at 6:30pm on "The Disabled Gaze: Rethinking the Past, Remaking the Future". 
+Please contact the instructors if you would like to attend -- we have 25 tickets to give out
+- In addition, she will join us in class for a conversation about disability and technology and the disabled gaze. This is a chance for you to bring questions about her and her work. 
 
-**Slides** {% include slide.html title="Guest Lecture: Avery Mack: Assessing Accessibility" loc="evaluation.html" %}
+**Friday Slides** {% include slide.html title="Designing for and with people with disabilities" loc="designing.html" %}
 
 {: .homework} Required Reading and Reflection (for Wednesday)
 : - Attend [**Jaipreet Virdi's talk**](https://www.washington.edu/lectures/events/the-disabled-gaze-rethinking-the-past-remaking-the-future/) at 6:30pm on Tuesday if you're able (ask us for tickets) *or* read [Q&A with Jaipret Virdi](https://disabilityvisibilityproject.com/2020/08/10/qa-with-jaipreet-virdi/) by Alice Wong on the [Disability Project website](https://disabilityvisibilityproject.com).
@@ -248,31 +237,12 @@ The lecture will be accompanied by an ASL interpreter and will include CART capt
   - Read [Anticipate and Adjust](https://dl.acm.org/doi/abs/10.1145/3491102.3501882)
   - Read [Accessibility came by accident](https://dl.acm.org/doi/pdf/10.1145/3173574.3174033?casa_token=T1I2RwihIjsAAAAA:QEm3SjurdlcW7oX_1LadxaglZ7oneBX-XLIjMCHbof3gu_IFpDW2OO5tqxZfLIps-94Qik9y5wNw8Q)
   - Read [How to do something good in the disability community if you're not disabled](https://www.forbes.com/sites/andrewpulrang/2020/12/16/how-to-do-something-good-in-the-disability-community-if-youre-not-disabled/?sh=21ef25c97d7f)
-  
-
-{% enddetails %}
-
-{% details Wednesday 10/25: Conversation with Jaipreet Virdi %} 
-- Dr. Jaipreet Virdi will be lecturing on 10/24 at 6:30pm on "The Disabled Gaze: Rethinking the Past, Remaking the Future". 
-Please contact the instructors if you would like to attend -- we have 25 tickets to give out
-- In addition, she will join us in class for a conversation about disability and technology and the disabled gaze. This is a chance for you to bring questions about her and her work. 
 
 {: .homework} Homework
 : [Technology Review](assignments/technology-review.html) Assigned: Write a group report about your assessment
 
-
-
-
-{% enddetails %}
- 
-{% details Thursday: Section %} 
-TBD
-{% enddetails %}
-
-{% details Friday 10/27: Introduction to Designing for Accessibility  %} 
-
-**Slides** {% include slide.html title="Designing for and with people with disabilities" loc="designing.html" %}
-
+{: .homework} **Post-module Reflection:**  [Accessibility Basics Reflection](https://forms.gle/Wk8C51m5Sbq8RX7W8)
+: 
 {% enddetails %}
 
 

slides/best-practices.html

@@ -22,6 +22,12 @@ class:
 ## This is an important reminder
 ## Make sure zoom is running and recording!!!
 
+---
+# Announcements
+
+My office hours on Tuesday will be at 4pm instead of 3pm
+
+
 ---
 [//]: # (Outline Slide)
 # Learning Goals for Today 
@@ -34,9 +40,9 @@ Building for Accessibility
 - Structure impacts navigation order
 - Need to announce things that change
 
+<!-- [//]: # TODO expand this slide deck? IT's short and could cover more. Also discuss how this should impact the report -->
 
 ---
-[//]: # TODO expand this slide deck? IT's short and could cover more. Also discuss how this should impact the report
 
 # Why isn't the World Already Accessible?
 
@@ -51,11 +57,6 @@ Testing accessibility is also hard!
 
 **Organizations impact accessibility**
 
-Designs have a big role in what is accessible
-
-Programmers also of course very important
-
-These days, a lot of it is created by end users
  
 ---
 # Who Creates Accessibility?
@@ -64,9 +65,6 @@ Organizations impact accessibility
 
 **Designs have a big role in what is accessible**
 
-Programmers also of course very important
-
-These days, a lot of it is created by end users
 
 ---
 # How might UX Designers address Accessibility
@@ -84,29 +82,47 @@ Organizations impact accessibility
 
 Designs have a big role in what is accessible
 
-**Programmers also of course very important**
+**Developers also of course very important**
 - Need to understand the expectations of APIs and accessibility technologies
 - Need to understand screen readers
 
-These days, a lot of it is created by end users
 
 ---
-[//]: # TODO fill in
-# What do Programmers already know about access?
+# How do practitioners enact accessibility in practice?
 
 [Accessibility in Software Practice](https://dl.acm.org/doi/pdf/10.1145/3503508)
 
-???
-Summarize this more...
+- data from 15 interviews and 365 survey respondents from 26 countries across five continents --> 120+ unique codes
+- followed up with a survey --> 44 statements grouped into eight topics on accessibility from practitioners’ viewpoints and different software development stages. 
+
 
 ---
-[//]: # TODO fill in
-# Organizational Issues
+# Organizational & People Challenges 
 
-<!-- --- -->
-<!-- # How might Designers address Accessibility -->
+.quote[Before making any decisions about “Accessibility”: stakeholders (e.g., designers, architects, developers, testers, and clients) in a project should
+reach a consensus on accessibility development and design]
 
-<!-- <iframe src="https://embed.polleverywhere.com/free_text_polls/sL5v5Ufo0sHFBmmC15MPV?controls=none&short_poll=true" width="800px" height="600px"></iframe> -->
+| Challenge                              | Recommendation                                     |
+|----------------------------------------|----------------------------------------------------|
+| Lack of resources                      | Long-term organizational buy-in and budget         |
+| Culture                                | Cooperative Culture                                |
+| Size (too small)                       | Work with customers & teams to prioritize access   |
+| Inadequate expertise & education       | Include accessibility expertise among team members |
+| Lack of QA to go with developer effort | Include accessibility on testing team              |
+
+---
+# Process Challenges (technical)
+
+Notice details of WCAG guidelines low on this list!
+
+| Challenge                                 | Recommendation                                       |
+|-------------------------------------------|------------------------------------------------------|
+| Unclear requirements & planning           | Include accessibility at all stages                  |
+| Unclear scope & architecture requirements | Engage with relevant end users                       |
+| Difficulty testing                        | Use appropriate testing suites & integration testing |
+| Lack of complete access practices         | Rigorous refactoring                                 |
+| Innapropriate tools                       | Well-designed documentation    & training            |
+| Domain-dependent issues                   | Appropriate end user engagement and testing                                                     |
 
 ---
 # Who Creates Accessibility?
@@ -115,11 +131,10 @@ Organizations impact accessibility
 
 Designs have a big role in what is accessible
 
-Programmers also of course very important
+Developers also of course very important
 
 **These days, a lot of it is created by end users**
 - This means that you have to think about *indirect* impacts on content creation too (i.e. what do you expose to end users in authoring tools)?
-- Will talk more about this next week, but crowdsourcing & online social networks part of this too
 
 ---
 [//]: # (Outline Slide)
@@ -133,7 +148,6 @@ Building for Accessibility
 - Structure impacts navigation order
 - Need to announce things that change
 
-
 ---
 # (On-desktop) screen reader interaction
 Three core  interaction patterns:

slides/comparing-approaches.html

 ---
 layout: presentation
-title: FOOBAR  --Week N--
+title: Comparing Assessment Techniques
 description: Accessibility
 class: middle, center, inverse
 ---
@@ -9,7 +9,7 @@ background-image: url(img/people.png)
 .left-column50[
 # Welcome to the Future of Access Technologies
 
-Week N, FOOBAR
+Comparing Assessment Techniques
 
 {{site.classnum}}, {{site.quarter}}
 ]
@@ -61,8 +61,7 @@ How do you get a system to the point where user testing is worth doing?
 
 [Is your web page accessible? A comparative study...](https://dl.acm.org/doi/10.1145/1054972.1054979)
 
-Gather baseline problem data on 4 sites
-- Usability study
+Gather baseline problem data on 4 sites (Usability Study)
 
 Test same sites with other techniques
 - Expert review with guidelines
@@ -101,11 +100,11 @@ AT the time, WCAG 1; Meeting WCAG priority 1 guidelines did not address all seve
 # Results -- Grocery
 
 .left-column50[
-![:img Picture of the front page of the albertson's website for ordering groceries online,100%, width](img/assessment/grocery.png) 
+![:img Picture of the front page of the albertson's website for ordering groceries online,80%, width](img/assessment/grocery.png) 
 ]
 
 .right-column50[
-![:img Picture of the grocery cart for the the albertson's website,100%, width](img/assessment/grocery2.png) 
+![:img Picture of the grocery cart for the the albertson's website,83%, width](img/assessment/grocery2.png) 
 ]
 ---
 # Results -- Grocery
@@ -124,7 +123,7 @@ Easiest site
 ]
 
 .right-column50[
-![:img Picture of the grocery cart for the the albertson's website,100%, width](img/assessment/grocery2.png) 
+![:img Picture of the grocery cart for the the albertson's website,83%, width](img/assessment/grocery2.png) 
 ]
 ---
 # Results -- Find Names
@@ -139,7 +138,7 @@ Easiest site
 ]
 
 .right-column50[
-![:img Picture of the list of graduate students in Berkeley's HCI group GUIR at the time of the study,100%, width](img/assessment/findnames.png) 
+![:img Picture of the list of graduate students in Berkeley's HCI group GUIR at the time of the study,80%, width](img/assessment/findnames.png) 
 ]
 
 ---
@@ -174,7 +173,7 @@ Most difficult site
 ]
 
 .right-column50[
-![:img Picture of a simple fake class registration form we made for the study,100%, width](img/assessment/registration.png) 
+![:img Picture of a simple fake class registration form we made for the study,80%, width](img/assessment/registration.png) 
 ]
 
 ---
@@ -243,32 +242,41 @@ No correlation between developer severity and WCAG priority or empirical severit
 
 ---
 # H1: Methods Don't Differ 
-.left-column50[
-- Screen reader and Expert Review found more problems
-]
-.right-column50[
-![:img same graph highlighting that the average reviewer only found less than 20% of problems,100%, width](img/assessment/graph2.png) 
+.left-column[
+Manual Review found lots of porblems
 ]
-???
+.right-column[
+<!-- <div class="mermaid"> -->
+<!-- pie title Problems Found by Condition -->
+<!--     "Dev. Review" : 8 -->
+<!--     "Guidelines Only" : 10 -->
+<!--     "Remote" : 9 -->
+<!-- </div> -->
 
-Note small differences between individual developers in finding problems
+![:img barchart showing that Dev. Review found 8 problems, Guidelines 10, and Remote 9,100%, width](img/assessment/IDCondition.png) 
 
-Difference between remote and screen reader group is significant
+]
 
 ---
 # H1: Methods Don't Differ 
-.left-column50[
-- Screen reader and Expert Review found more problems
-- Screen reader and Expert Review most valid
+.left-column[
+Manual Review as effective as remote screen reader users: % of problems reported in each condition that matched known problems
 ]
-.right-column50[
-![:img same graph highlighting that the average validity is 60% for remote BLV users; 20% for expert reviewers; and 40% for non-BLV screen reader users,100%, width](img/assessment/validity.png) 
+
+.right-column[
+![:img barchart showing that Dev. Review 78% valid, Guidelines 94% valid, and Remote 82% valid,80%, width](img/assessment/Validity.png) 
 ]
-???
 
-Note small differences between individual developers in finding problems
+---
+# H1: Methods Don't Differ 
+.left-column[
+Manual Review as effective as remote screen reader users: % of known accessibility problems found in each condition
+]
+
+.right-column[
+![:img barchart showing that Dev. Review 33% valid, Guidelines 41% valid, and Remote 26% valid,80%, width](img/assessment/Thoroughness.png) 
+]
 
-Difference between remote and screen reader group is significant
 
 ---
 # H2: Techniques find Different Problems
@@ -335,27 +343,27 @@ Many (perhaps all) of these are part of guidelines now
 - E8: Poor names
 - E9: Popups
 
----
-# H2: Techniques find Different Types of Problems
+<!-- --- -->
+<!-- # H2: Techniques find Different Types of Problems -->
 
-- High variance among individual reviewers
-- Screen reader novices did best at both major types of problems
+<!-- - High variance among individual reviewers -->
+<!-- - Screen reader novices did best at both major types of problems -->
 
-![:img Four bar charts each showing the cumulative benefit in terms of percentage of known problems found of adding evaluators. The first bar chart shows expert reviewers; who reach 30% of WCAG problems and 60% of empirical problems by the third evaluator. The second bar chart shows novice screen reader users who reach 60% of both empirical and WCAG problems by the fifth evaluator. The third bar chart shows remote BLV participants who reach 25% of WCAG and just under 20% of empirical problems by the fifth evaluator. The fourth bar chart shows an automated tool which finds about 25% of WCAG and 5% of empirical problems,100%, width](img/assessment/cumulative.png) 
+<!-- ![:img Four bar charts each showing the cumulative benefit in terms of percentage of known problems found of adding evaluators. The first bar chart shows expert reviewers; who reach 30% of WCAG problems and 60% of empirical problems by the third evaluator. The second bar chart shows novice screen reader users who reach 60% of both empirical and WCAG problems by the fifth evaluator. The third bar chart shows remote BLV participants who reach 25% of WCAG and just under 20% of empirical problems by the fifth evaluator. The fourth bar chart shows an automated tool which finds about 25% of WCAG and 5% of empirical problems,100%, width](img/assessment/cumulative.png)  -->
 
-???
-Explain chart
-also tracks heuristic eval literature: Five Evaluators find ~50% of Problems
-Individuals don't do well, but they  *differ*  from each other
+<!-- ??? -->
+<!-- Explain chart -->
+<!-- also tracks heuristic eval literature: Five Evaluators find ~50% of Problems -->
+<!-- Individuals don't do well, but they  *differ*  from each other -->
 
 ---
-# Discussion
+# Other findings
 
-Hyp 1: Screen reader most consistently effective
+<!-- Hyp 1: Screen reader most consistently effective -->
 
-Hyp 2: All but automated comparable
+<!-- Hyp 2: All but automated comparable -->
 
-- Screen missed only tables (w3); poor defaults (empirical)
+<!-- - Screen missed only tables (w3); poor defaults (empirical) -->
 
 Really need multiple evaluators
 
@@ -363,16 +371,16 @@ Remote technique needs improvement, could fare better
 
 Accessibility experience would probably change results
 
----
-# Discussion
+<!-- --- -->
+<!-- # Discussion -->
 
-Asymptotic testing needed
-- Can’t be sure we found all empirical problems
+<!-- Asymptotic testing needed -->
+<!-- - Can’t be sure we found all empirical problems -->
 
-Falsification testing needed
-- Are problems not in empirical data set really false positives?
+<!-- Falsification testing needed -->
+<!-- - Are problems not in empirical data set really false positives? -->
 
-More consistent problem reporting & comparison beneficial
+<!-- More consistent problem reporting & comparison beneficial -->
 
 Limitations
 - Web only