-
Dhruv authoreda1de1da8
bias-in-machine-learning.html 12.28 KiB
---
layout: presentation
title: AI Accessibility Bias --Week 7--
description: Discussion of Bias in AI
class: middle, center, inverse
---
background-image: url(img/people.png)
.left-column50[
# Week 7: Automating Accessibility
{{site.classnum}}, {{site.quarter}}
]
---
name: normal
layout: true
class:
---
# Important Reminder
## This is an important reminder
## Make sure zoom is running and recording!!!
## Make sure captioning is turned on
---
[//]: # (Outline Slide)
# Learning Goals for Today
- How do AI & ML algorithms work?
- How do we minimize disability bias in automated systems?
- How do we collect data? Who do we collect the data from?
- How do we know whether the data is "good"?
---
# AI & Machine Learning
Both can change the way we think about a problem.
But *how*?
- What is the *traditional* approach to solve a problem?
- How does AI solve a problem?
---
# Helping Computers Learn Patterns
.left-column50[
![:fa bed, fa-7x]
]
.right-column50[
## How might you recognize sleep?
- Can you come up with a yes/no question or a set of categories or simple description of sleep?
- Sleep quality?
- Sleep start/end?
- What data would you learn from?
- How might you need to take disabilites into account?
]
???
(sleep quality? length?...)
How to interpret sensors?
---
# How do we program this?
Old Approach: Create software by hand
- Use libraries (like JQuery) and frameworks
- Create content, do layout, code up functionality
- Deterministic (code does what you tell it to)
New Approach: Collect data and train algorithms
- Will still do the above, but will also have some functionality based
on AI
- *Collect lots of examples and train a AI algorithm*
- *Statistical way of thinking*
---
# Shift in Approaches
.left-column50[
## Old style of app design
<div class="mermaid">
graph TD
I(Input) --Explicit Interaction--> A(Application)
A --> Act(Action)
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
class U,C,A,I,S,E,Act,Act2 normal
</div>
]
--
count: false
.right-column50[
## New style of app design
<div class="mermaid">
graph TD
U(User) --Implicit Sensing--> C(Application)
S(System) --Implicit Sensing--> C
E(Environment) --Implicit Sensing--> C
C --> Act2(Action)
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
class U,C,A,I,S,E,Act,Act2 normal
</div>
]
---
# Basic Approach Of All AI
- Collect data (and lots and lots of it!)
- Discern patterns
- Make predictions
---
# Pause and Discuss
How could disability bias affect these?
- Collect data (and lots and lots of it!)
- Discern patterns
- Make predictions
(Post on [Ed]({{site.discussion}}/3805159)
---
# Data Collection
- How do we collect data?
- Where do we collect data from?
- Who do we collect data from?
---
# Problems with Data (1/2)
- System timeouts that are trained on movement speeds of <q>typical</q> people
- Biometrics that cannot function on a person who isn't still for long enough
- Inferencing about people that doesn't account for height; stamina; range of motion; or AT use (e.g. wheelchairs)
---
# Problems with Data (2/2)
When groups are historically
marginalized and underrepresented, this is
.quote[imprinted in the data that shapes AI
systems... Those who have borne discrimination in the past are most at risk of harm from
biased and exclusionary AI in the present. (Whittaker, 2019)]
--
This can cascade -- e.g. measurement bias can exacerbate bias downstream. For example, facial mobility, emotion expression, and facial structure impact detection and identification of people; body motion and shape impact activity detection; etc.
---
# How might we address bias/fairness in data sets
We need to know it is there (Aggregate metrics can hide performance problems in under-represented groups)
We need to be careful not to eliminate, or reduce the influence, of outliers if that erases disabled people from the data because of the heterogeneity of disability data.
---
# Approaches to measuring fairness
We may need to rethink <q>fairness</q> in terms of individual rather than group outcomes, and define metrics that capture a range of concerns
- Movement speed might favor a wheelchair user
- Exercise variety might favor people who do not have chronic illness
- Exertion time might covers a wide variety of different types of people.
Defining such unbiased metrics requires careful thought and domain knowledge, and scientific research will be essential to defining appropriate procedures for this.
<!-- --- -->
<!-- # Small Group Discussion [Post on Ed]({{site.discussion}}TBD) -->
<!-- Who might be excluded in the data set you found? -->
<!-- How was fairness measured in the data set you found, if it was discussed? -->
<!-- How would you go about testing for fairness in that data? -->
---
# Best Practices For Data Fairness (1/2)
- How do we motivate and ethically compensate disabled people to give their data?
- What should we communicate at data collection time?
- Is the data collection infrastructure accessible? Does it protect sensitive information about participants adequately given the heterogeneous nature of disability?
---
# Best Practices For Data Fairness (2/2)
- Does the meta data collected oversimplify disability? Who is labeling the data and do the have biases affecting labeling?
- Whittaker (2019) discusses the example of clickworkers who label people
as disabled <q>based on a hunch</q>.
---
# Basic Approach Of All AI
- Collect data (and lots and lots of it!)
- **Discern patterns**
- Make predictions
---
# Two main approaches
![:fa eye] *Supervised learning* (we have lots of examples of what should be
predicted)
![:fa eye-slash] *Unsupervised learning* (e.g. clustering into groups and inferring what
they are about)
![:fa low-vision] Can combine these (semi-supervised)
![:fa history] Can learn over time or train up front
---
# Machine Learning
.left-column50[
## Training Process
<div class="mermaid">
graph TD
L(Label) --> MI(Training Algorithm)
D(Input Data) -- Extract Features--> MI
MI --> C(Symbolic Predictor)
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
class D,U,C,A,I,S,E,Act,Act2 normal
</div>
]
.right-column50[
## Extracting Features
Symbolic requires feature engineering (humans deciding how to *summarize* data using features. Tends to be more *interpretable* (you can figure out why they make predictions)
]
---
# Large Language Models
.left-column50[
## Training Process
<div class="mermaid">
graph TD
L(Label) --> MI(Training Algorithm)
D(Input Data) --> MI
MI --> C(Neural Predictor)
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
class D,U,C,A,I,S,E,Act,Act2 normal
</div>
]
.right-column50[
## Designing Networks
Neural approaches (e.g. ChatGPT) use massive amounts of data to train a network according to base principales. Designing the right network is critical. Cannot be sure *why* they make the predictions they do.
]
---
# Both
## Prediction Process
.left-column[
<div class="mermaid">
graph TD
D(Input Data) --> C(Symbolic/Neural Predictor)
C --> P(Prediction)
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
class P,D,U,C,A,I,S,E,Act,Act2 normal
</div>
]
---
# How do we Evaluate Predictors/Predictions?
Norms are baked deeply into algorithms which are designed to learn about the most common cases. As human judgment is increasingly replaced by AI, *norms* become more strictly enforced.
- Do outliers face higher error rates?
- Do they disproportionately represent and misrepresent people with disability?
- How does this impact allocation of resources?
---
# How does norming harm people with disabilities? (1/2)
Machine intelligence already being used to track allocation of assistive technologies, from CPAP machines for people with sleep apnea (Araujo 2018) to prosthetic legs (as described by Jullian Wiese in
Granta and uncovered in Whittaker et al 2019), deciding who is <q>compliant enough</q> to deserve them.
---
# How does norming harm people with disabilities? (2/2)
Technology may also fail to recognize that a disabled person is even present (Kane, 2020), thus <q>demarcating what it means to be a legible human and
whose bodies, actions, and lives fall outside... [and] remapping and calcifying the boundaries
of inclusion and marginalization</q> (Whittaker, 2019).
---
# How does norming harm people with disabilities? (3/2)
Many biometric systems gatekeep access based on either individual identity, identity as a human, or class of human, such as <q>old enough to buy cigarettes.</q>
Examples:
- a participant having to falsify data because <q>some apps [don’t allow] my height/weight combo for my age.</q> (Kane (2020))
- a person who must ask a stranger to ‘forge’ a signature at the grocery store <q>.. because I can’t reach [the tablet]</q> (Kane (2020))
- at work, activity tracking may define <q>success</q> in terms that exclude disabled workers. (may also increase the likelihood of work-related disability, by forcing workers to work at maximal efficiency)
---
# Basic Approach Of All AI
- Collect data (and lots and lots of it!)
- Discern patterns
- **Make predictions**
---
# Stopp and Discuss
What do you think could go wrong at prediction time?
Add a second (Post on [Ed]({{site.discussion}}/3805238)
---
# Concerns at Prediction Time (1/5)
- Denial of insurance and medical care, or threaten employment (Whittaker,
2019, p. 21).
- HireVue, an AI based video
interviewing company has a patent on file to detect disability (Larsen, 2018).
- This is illegal under the ADA, which
- forbids asking about disability
status in a hiring process (42 U.S.C. § 12112(a))
- forbids <q>using qualification
standards, employment tests or other selection criteria that screen out or tend to screen out
an individual with a disability</q> (42 U.S.C. § 12112(b)(6)).
---
# Concerns at Prediction Time (2/5)
- Denial of insurance and medical care, or threaten employment
- Disability identification
- Examples: detect Parkinsons from gait (Das, 2012), and mouse movement (Youngmann,
2019); detecting autism from home videos (Leblanc, 2020).
- What are the ethics of doing this without consent?
- Many of these algorithms encode medical model biases
- Relatedly, failure to identify disability
- Legally under the ADA, if you are treated as disabled, you are disabled. Yet biometrics cannot detect how people are treated.
---
# Concerns at Prediction Time (3/5)
- Denial of insurance and medical care, or threaten employment
- Disability Identification / Failure to Identify
- Apps that Harm
- Example: Training behaviors in <q>support</q> of autistic individuals without regard to debates about agency and independence of the target audience [Demo, 2017];
- As with regular accessibility apps, AI based aps can harm, be disability dongles, etc
- As with regular apps, AI based apps may not be accessible
---
# Concerns at Prediction Time (4/5)
- Denial of insurance and medical care, or threaten employment
- Disability Identification / Failure to Identify
- Apps that Harm
- AI with Baked in Biases
- Consequences of biased data and lack of control over training results more nuanced than just accuracy (as with headlines we just read)
- Privacy can also be a concern.
- For rare conditions, an algorithm may learn
to recognize the disability, rather than the individual, reducing
security when used for access control, allowing multiple people with
similar impairments to access the same data.
---
# Concerns at Prediction Time (5/5)
- Denial of insurance and medical care, or threaten employment
- Disability Identification / Failure to Identify
- Apps that Harm
- AI with Baked in Biases
- Transparency and Accountability
- Power differences between builders and users
- Representation of disabled people among builders
- Algorithms that are not *interpretable* or *correctable*
- Users of algorithms whose use them to enforce larger societal harms
<!-- --- -->
<!-- # Small Group Discussion [Post on Ed]({{site.discussion}}2515387) -->
<!-- Revisit the data set you chose -->
<!-- Do you know what sort of predictions it was used for if any? -->
<!-- What possible harms could be done with those predictions? -->
<!-- Reminder of our list --><!-- - Denial of insurance and medical care, or threaten employment -->
<!-- - Disability Identification / Failure to Identify -->
<!-- - Apps that Harm -->
<!-- - AI with Baked in Biases -->
<!-- - Transparency and Accountability -->
---
# End of Deck