Data Equity First Draft

schedule.md

@@ -282,20 +282,92 @@ Discussion of Disability Dongle Paper
 {: .homework} Readings for next week 
 : 
 
+{% details Preparation for next week  %}
+**Required: Respond to the Reading Questions and Preparation Requirements on Ed**
+
+- [Areas of Strategic Visibility: Disability Bias in Biometrics](https://arxiv.org/abs/2208.04712)
+- [Increasing Data Equity in Accessibility](https://arxiv.org/abs/2210.01902)
+- [Understanding Screen-Reader Users’ Experiences with Online Data Visualizations](https://athersharif.me/documents/assets-2021-understanding-sru-experiences-online-data-viz.pdf)
+
+**Optional**
+- [The Future of Urban Accessibility for People with Disabilities: Data Collection, Analytic, Policy, and Tools](https://dl.acm.org/doi/fullHtml/10.1145/3517428.3550402)
+- [Explaining Explanations: An Approach to Evaluating Interpretability of Machine Learning](https://www.cs.purdue.edu/homes/bb/nit/Lalana-Explainations%20of%20IAS.pdf)
+- [Rich Screen Reader Experiences for Accessible Data Visualization](https://arxiv.org/pdf/2205.04917.pdf)
+- [VoxLens: Making Online Data Visualizations Accessible With an Interactive JavaScript Plug-In](https://athersharif.me/documents/chi-2022-voxlens.pdf)
+
+{% enddetails %}
+
 {: .week}
 # Week 6 (2/7): Data Equity
-**Learning Goals** TBD
+{% details Learning Goals %} 
+- Bias in Machine Learning
+    - What is Machine Learning (ML)? 
+    - What are the components of ML?
+    - How do we collect data? Who do we collect the data from?
+    - Is the data "good"?
+    - How do we minimize disability bias?
+- Accessible Data Visualizations
+    - What are the commonly-used techniques to make data visualizations accessible?
+    - What are the pros and cons of these techniques?
+    - What are some of the nuances in making data visualizations accessible?
+{% enddetails %}
 
-{% details Class Plan: STILL TBD %} 
-...
+{% details Class Plan %} 
 
-{% enddetails %}
+{: .activity} 5:30-5:35 Class Introductions (Instructor - [Ather Sharif](https://athersharif.me) [he/him])
+: 
 
+{: .topic}  5:35-6:00 Bias in Machine Learning
+: **Slides** {% include slide.html title="Bias in Machine Learning" loc="bias-in-machine-learning.html" %}
+
+{: .activity} 6:00-6:20 Discussion
+: 
+- Identify biases in existing ML-based tools/platforms
+- Identify ways to make ML models more inclusive in your area of expertise
+
+{: .break}  6:20-6:30 Break / Food 
+: 
+
+{: .topic}  6:30-7:00 Accessible Data Visualizations
+: **Slides** {% include slide.html title="Accessible Data Visualizations" loc="accessible-data-viz.html" %}
+
+{: .activity} 7:00-7:40 Skill Goal: Various Techniques to Make Visualizations Accessible
+: **In class Exercise**
+- Implement a visualization using [VoxLens](https://github.com/athersharif/voxlens) following the documentation and examples on the GitHub repository.
+- Use VoxLens from to test different modes of making visualizations accessible (Summary, Sonification, Question-and-Answer) on different visualization types.
+- Identify the utility of each mode with each visualization type–when is one mode more suitable than the other and why.
+- Identify accessibility measures for demographics other than screen-reader users.
+- Create a presentation (2-3 slides) to share findings
+
+{: .break} 7:45-7:55 Break 
+: 
+
+{: .activity} 7:55-8:25 Group Presentations 
+: Your presentation should demonstrate your experiences with using the different techniques used to make data visualizations accessible and the benefits of each technique. Is any particular technique better suited than the others when considering visualizations types? What other techniques are worth exploring?
+
+{: .activity} 7:40-7:45 Reflection Writing
+: 
+- Elaborate further on different techniques used and discuss the pros and cons for each technique
+- Identify unconventional techniques that can improve the accessibility of data visualizations
+- Identify how ML can improve the accessibility of data visualizations
+
+{: .break} 8:25-8:30 Wrap-up 
+: 
+
+{% enddetails %}
 
 {: .homework} [Class Participation survey](https://forms.gle/qcjc5fSNSAVeE5pK9)
-{: .homework} Readings for next week 
 : 
 
+{% details Preparation for next week  %}
+**Required: Respond to the Reading Questions and Preparation Requirements on Ed**
+- TBD
+
+**Optional**
+- TBD
+
+{% enddetails %}
+
 {: .week}
 # Week 7 (2/14): Making Accessibility: Fabrication and Physical Computing
 **Learning Goals** TBD

slides/accessible-data-viz.html

+---
+layout: presentation
+title: Accessible Data Visualizations  --Week 6--
+description: Comparison of Assessment Approaches
+class: middle, center, inverse
+---
+background-image: url(img/people.png)
+
+.left-column50[
+# Week 6: Accessible Data Visualizations
+
+{{site.classnum}}, {{site.quarter}}
+]
+---
+name: normal
+layout: true
+class:
+
+---
+# Important Reminder
+
+## This is an important reminder
+## Make sure zoom is running and recording!!!
+
+---
+[//]: # (Outline Slide)
+# Learning Goals for Today
+
+- What are the commonly-used techniques to make data visualizations accessible?
+
+- What are the pros and cons of these techniques?
+
+- What are some of the nuances in making data visualizations accessible?
+
+---
+# COVID-19 Visualizations
+
+![:img Visualizations of COVID-19 cases taken from different websites including Seattle Times and King County’s official website,100%, width](img/data-equity/covid-19-viz.png)
+
+---
+# How Can We Make These Visualization Accessible?
+
+- What strategies and techniques **do** you use to make these visualizations accessible to screen-reader users?
+
+- What strategies and techniques **could** you use to make these visualizations accessible to screen-reader users?
+
+---
+# Commonly-Used Techniques
+
+- Summary ("Alt-Text")
+
+- Sonification (Audio Graphs)
+
+- Tables
+
+- Braille Prinouts/3-D Printing
+
+---
+# Summary ("Alt-Text")
+
+Alt-text is an HTML attribute that can provide a summary of the visualization and the information it contains.
+
+**Benefits**
+- Most commonly-used strategy
+
+- Easy to integrate into web elements
+
+---
+# Sonification (Audio Graphs)
+
+Sonification (or audio graphs) enable listeners to interpret data trends by mapping data to a musical scale.
+
+[Example](https://www.youtube.com/watch?v=gssjuSnMR60)
+
+**Benefits**
+- Can help users to understand overall trends
+
+- Already integrated into commerical products (e.g., Apple)
+
+---
+# Tables
+
+Tables are HTML elements that can be used to convey data to users.
+
+**Benefits**
+- Provide users access to raw data 
+
+- Already integrated into Google Charts
+
+---
+# Tactile Graphics/3-D Printing
+
+Tactile graphics and 3-D printed graphs are raised surfaces that enable blind and low vision individuals to interpret data by feeling the graphs. 
+
+**Benefits**
+- Can assist users in developing a mental model of the visualization
+
+- Enables users to interact with the visualization without using a screen reader
+
+---
+# Equality vs. Equity
+
+- What does **equal** access mean?
+
+- What does **equitable** access mean?
+
+- Are these techniques equitable?
+
+---
+# State of The World
+
+Due to the inaccessibility of online data visualizations, screen-reader users answered questions **62%** less accurately and spent **211%** more time than non-screen-reader users *(Sharif, 2021)*.
+
+![:img Two different graphs comprising bar chart of Accuracy of Extracted Information per Visualization Library and interaction plot of Interaction Times per Visualization Library per participant group.,100%, width](img/data-equity/sru-viz-results.png)
+
+---
+# Holistic vs. Drilled-Down Exploration
+
+Screen-reader users perform a holistic exploration first and then drill-down into the data *(Sharif, 2021)*.
+
+- Which of these techniques enable **holistic** exploration?
+
+- Which of these techniques enable **drilled-down** exploration?
+
+---
+# VoxLens 
+
+VoxLens is an open-source JavaScript plug-in that improves the accessibility of online data visualizations for screen-reader users using a multi-modal approach, requiring only a single line of code for integration into online data visualizations. 
+
+VoxLens offers three modes: 
+
+- **Question-and-Answer mode**
+- **Summary mode**
+- **Sonification mode**
+
+.center[https://github.com/athersharif/voxlens]
+
+---
+# Evaluation of VoxLens
+
+VoxLens improved the accuracy of screen-reader users by **122%** and interaction times of screen-reader users by **36%**.
+
+![:img Two different graphs comprising bar chart of Accuracy of Extracted Information per Visualization Library and interaction plot of Interaction Times per Visualization Library per participant group.,100%, width](img/data-equity/voxlens-viz-results.png)
+
+---
+# Class Activity
+
+**Activitiy**
+- Break into small groups.
+- Implement a visualization using [VoxLens](https://github.com/athersharif/voxlens).
+- Use VoxLens from to test different modes of making visualizations accessible (Summary, Sonification, Question-and-Answer).
+
+**Goals**
+- Identify the utility of each mode with each visualization type–when is one mode more suitable than the other and why.
+- Identify accessibility measures for users other than screen-reader users.
+- Create a presentation (2-3 slides) to share findings
+
+.center[https://github.com/athersharif/voxlens]
+
+---
+[//]: # (Outline Slide)
+# Learning Goals for Today
+
+- What are the commonly-used techniques to make data visualizations accessible?
+
+- What are the pros and cons of these techniques?
+
+- What are some of the nuances in making data visualizations accessible?
+

slides/bias-in-machine-learning.html

+---
+layout: presentation
+title: Bias in Machine Learning  --Week 6--
+description: Comparison of Assessment Approaches
+class: middle, center, inverse
+---
+background-image: url(img/people.png)
+
+.left-column50[
+# Week 6: Bias in Machine Learning
+
+{{site.classnum}}, {{site.quarter}}
+]
+---
+name: normal
+layout: true
+class:
+
+---
+# Important Reminder
+
+## This is an important reminder
+## Make sure zoom is running and recording!!!
+
+---
+[//]: # (Outline Slide)
+# Learning Goals for Today
+
+- What is Machine Learning (ML)? 
+
+- What are the components of ML?
+
+- How do we collect data? Who do we collect the data from?
+
+- Is the data "good"?
+
+- How do we minimize disability bias?
+
+---
+# Machine Learning
+
+![:img Screenshots of recent news articles on machine learning,100%, width](img/data-equity/ml-news.png)
+
+.center[**But really, *what is it*?**]
+
+---
+# Machine Learning
+
+Machine Learning changes the way we think about a problem.
+
+But *how*?
+
+- What is the *traditional* approach to solve a problem?
+
+- How does Machine Learning solve a problem?
+
+---
+# Components of Machine Learning
+
+- Collect data (and lots and lots of it!)
+
+- Discern patterns
+
+- Make predictions
+
+---
+# Data Collection
+
+- How do we collect data?
+
+- Where do we collect data from?
+
+- Who do we collect data from?
+
+---
+# "Goodness" of Data
+
+Let's consider this example: **We wish to train a ML model to recognize if a given picture is that of a nurse.**
+
+What steps do we take?
+
+Is the data "good"?
+
+What other examples can we think of?
+
+---
+# "Goodness" of Data
+
+![:img Confusion matrix of a machine learning model,100%, width](img/data-equity/ml-faulty.png)
+
+---
+# Disability Bias in Machine Learning 
+
+Data sets used to train biometric systems are biased: they rarely if ever have a comprehensive
+representation of the range of people they might encounter. *(Mankoff, 2021)*
+
+- Human bias 
+
+- Machine bias
+
+- Underrepresentation
+
+- Measurement Errors
+
+---
+# Minimizing Bias
+
+- Appropriate representation
+
+- Large training data 
+
+- Validating the models 
+
+- Montoring the performance of the models
+
+---
+# Discussion
+
+- Identify biases in existing ML-based tools/platforms
+
+- Identify ways to make ML models more inclusive in your area of expertise