Skip to content
Snippets Groups Projects
Commit 91ce36b7 authored by Jennifer Mankoff's avatar Jennifer Mankoff
Browse files

A lot of changes to the main website as well as class materials for this week.

parent 2702b681
No related branches found
No related tags found
6 merge requests!12Accessibility website jen summer work,!9Q access 24 sp,!8Q access 24 sp,!5Latest update from Spring 2023,!4Latest update after pmp class,!3Updated content for week 3 and week 4 (week 4 still needs work)
Showing
with 736 additions and 238 deletions
......@@ -73,8 +73,7 @@ sass:
kramdown:
toc_levels: '1'
mermaid:
src: 'https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js'
mermaid: 'https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js'
remarkConfig:
# possible highlight options:
......
......@@ -69,8 +69,7 @@ sass:
kramdown:
toc_levels: '1'
mermaid:
src: 'https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js'
mermaid: 'https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js'
remarkConfig:
# possible highlight options:
......
......@@ -10,5 +10,7 @@ nav_list:
url: canvas
- title: Ed
url: discussion
- title: COVID-19 Safety
- title: COVID-19
url: /docs/covid/index.html
- title: Academic Conduct
url: /academic-conduct.html
\ No newline at end of file
<header class="page-header" role="banner">
{% include navigation.html %}
<br> <br> <br> <br> <br>
<br> <br> <br> <br>
<h1 class="project-name">{{ site.title | default: site.github.repository_name }}</h1>
<h2 class="project-tagline">{{ site.description | default: site.github.project_tagline }}</h2>
<script type="text/javascript">
var menubar = new Menubar(document.getElementById('menubar1'));
menubar.init();
......
......@@ -55,7 +55,7 @@
{% include footer.html %}
</main>
<script type="text/javascript" src="{{'assets/js/mermaid.8.8.2.js' | relative_url}}"></script>
<script type="text/javascript" src="{{site.mermaid}}"></script>
<link rel="stylesheet" href="{{ '/assets/css/mermaid.css' | relative_url }}">
<link rel="stylesheet" href="{{ '/assets/css/assignment.css' | relative_url }}">
<script>
......
......@@ -24,7 +24,7 @@
<script src="{{ '/assets/js/bootstrap.min.js' | relative_url}}"></script>
<script type="text/javascript" src="{{'assets/js/mermaid.8.8.2.js' | relative_url}}"></script>
<script type="text/javascript" src="{{site.mermaid}}"></script>
<link rel="stylesheet" href="{{ '/assets/css/mermaid.css' | relative_url }}">
<script>
var config = {
......
......@@ -45,7 +45,7 @@
<script type="text/javascript" src="{{'assets/js/mathjax-sre.js' | relative_url}}"></script>
<script type="text/javascript" src="{{'assets/js/html2canvas.min.js' | relative_url}}"></script>
<script type="text/javascript" src="{{'assets/js/presentation-export.js' | relative_url}}"></script>
<script type="text/javascript" src="{{'assets/js/mermaid.8.8.2.js' | relative_url}}"></script>
<script type="text/javascript" src="{{site.mermaid}}"></script>
<script type="text/javascript">
var slideshow = remark.create({
......
......@@ -45,7 +45,7 @@
<script type="text/javascript" src="{{'assets/js/mathjax-sre.js' | relative_url}}"></script>
<script type="text/javascript" src="{{'assets/js/html2canvas.min.js' | relative_url}}"></script>
<script type="text/javascript" src="{{'assets/js/presentation-export.js' | relative_url}}"></script>
<script type="text/javascript" src="{{'assets/js/mermaid.8.8.2.js' | relative_url}}"></script>
<script type="text/javascript" src="{{site.mermaid}}"></script>
<script type="text/javascript">
var slideshow = remark.create({
......
......@@ -3,22 +3,21 @@ layout: default
(no)warning: old
---
---
Integrity is a crucial part of your character and is essential for
a successful career. We expect you to demonstrate integrity in
CSE 154 and elsewhere.
The Paul G Allen School has an entire page on
[Academic Misconduct](https://www.cs.washington.edu/academics/misconduct)
within the context of Computer Science, and the University of
Washington has an entire page on how
[Academic Misconduct](https://www.washington.edu/cssc/for-students/academic-misconduct/)
[Community Standards and Student Conduct Page](https://www.washington.edu/cssc/).
Please acquaint yourself with both of those pages, and in
particular how academic misconduct will be reported to the
University.
this class and elsewhere.
The Paul G Allen School has an [Academic
Misconduct](https://www.cs.washington.edu/academics/misconduct)
policy, and the University of Washington an [Academic
Misconduct](https://www.washington.edu/cssc/for-students/academic-misconduct/)
and [Community Standards and Student Conduct
Page](https://www.washington.edu/cssc/) policy. Please acquaint
yourself with those pages, and in particular how academic misconduct
will be reported to the University. Knowingly violating any of these principles of academic conduct,
privacy or copyright may result in University disciplinary action
under the Student Code of Conduct.
Your academic conduct in this course is evaluated in at least the
four areas described in detail below.
......@@ -26,160 +25,110 @@ four areas described in detail below.
* TOC
{:toc}
# Honesty in Communications
# Honesty and Respect in Communications
Individuals are expected to be honest and forthcoming
in communications with TAs and the instructor.
in communications with TAs and the instructor.
In addition, individuals are expected to show respect for the
intellectual contributions of others through citation. The essence of
academic life revolves around respect not only for the ideas of
others, but also their rights to those ideas. It is therefore
essential that we take the utmost care that the ideas (and the
expressions of those ideas) of others always be handled appropriately,
and, where necessary, cited. This is an issue of [Citational
Justice](nature.com/articles/d41586-022-00793-1), and a core value of
this course.
When ideas or materials of others are
used (particularly in your creative projects), they must be cited. The
citation format is not that important - as long as the source material
can be located and the citation verified, it's OK. In any situation,
if you have a question, please feel free to ask. Here are some examples of how you might use (and cite) different types of content:
- Media you have created or generated yourself (i.e. pictures you have created or taken yourself, text you have written yourself) do generally not require citation. However if you have published them (on a blog, in an article etc), they may belong to the publisher and require citation.
- Images that are in the public domain (something from Wikipedia), or with a creative commons license that allows for reuse without explicit permission of the owner, require citation based on their license. Instructions on how to search for images that are fair use are [here](https://support.google.com/websearch/answer/29508?hl=en). [Creative Commons Kiwi](https://creativecommons.org/about/videos/creative-commons-kiwi/) is a really informative video on Creative Commons licensing; and here are [best practices](https://wiki.creativecommons.org/wiki/Best_practices_for_attribution) for citing Creative Commons works.
- If you are writing some text (a paper, analysis, etc), you can use quotes as long as you give attribution to the sources of the quote. You can not write an entire document out of the quotes of others, or include copied text with no citations. The citation format is not that important - as long as the source material can be located and the citation verified (a url in a comment is generally fine), it's OK.
- If you are writing code, clearly indicate (e.g. with comments) which portions of your code are completely original and which are used or modified from external sources, if any code is used that builds off of/is inspired by external sources (e.g. adaption of an example exercise, online tutorial you find). Note that solely changing identifier names or rearranging other source material is not considered your original work - see the examples of appropriate use below for details.
# Application Content
## School Appropriateness of Content
Some examples of appropriate use:
- A student finds a blog post explaining why and how to address WCAG guidelines. They rewrite much of it in their own words, but for a few sections that are particularly clear, they copy the information, put it in quotations, and name the source and provide a link to it right next to the quote.
- A student closely follows a tutorial to understand a new concept in Android Development (e.g. animations). The student cites the tutorial they used in the file header then substantially modifies the tutorial code to include what is specified for the creative portion of the assignment, documenting which portions of the code are their own so TAs know which portions to grade (and to determine whether the material cited as being learned from the tutorial is sufficiently adapted to be considered the student’s own work).
- A student is having difficulty generating audio from text for an accessibility feature of an app they are building. They look for a solution and find an app very similar to the one they intend to build. They fork it, and modify it for their final project, and documents this with a comment that includes where it was found. When grading the app, the instructional staff may weight features that were provided by the original source less than features that the student added themselves.
Recall that one of our course policies is to engender an
inclusive environment. As such it is important that you are thoughtful about
what you choose to post on your page. Please make sure that the
images and text you are using are “school appropriate” and follow
the guidelines of expected behavior. If you have any questions,
please do not hesitate to ask a TA or your instructor. Inappropriate work
submitted may be ineligible for credit on that assignment.
Students with questions about any specific situation should ask the instructor for clarification.
# Collaboration Policies
## Copyright and Citations
In this class, are encouraged to discuss class material, including
assignments, lecture material and readings with your classmates. Even
better if this takes place on Ed where other students can benefit and
we can guide you as to what is supportive and what crosses the line to
too much sharing.
Some assignments are individual. Even when as assignment is
individual, you **may discuss homework assignments with other
students** (i.e. provide advice, brainstorm) as long your writing
and/or implementation is entirely your own, and you document what you
do. You may also look at other sources online to learn how to achieve
new things, but we expect you to document this, and it may impact the
credit you get for your work. You should never copy (plagiarize) from
another person in this school (past or present) or from material that
you find online directly and submit it as your own work.
To facilitate this, and to be very clear
- You **may not use code or writing directly from any external
sources** (including copying lecture/lab material directly into an
assignment) without appropriately crediting the source as described above in <q>Honesty and Respect in Communications</q>
- You must credit a classmate when their advice had a significant intellectual impact on what you did
# Privacy and Fair Use
All of the expressions of ideas in this class that are fixed in
any tangible medium such as digital and physical documents are
To support an academic environment of rigorous discussion and open
expression of personal thoughts and feelings, we, as members of the
academic community, must be committed to the inviolate right of
privacy of our student and instructor colleagues. As a result, we must
forego sharing personally identifiable information about any member of
our community including information about the ideas they express,
their families, lifestyles and their political and social
affiliations. If you have any questions regarding whether a disclosure
you wish to make regarding anyone in this course or in the university
community violates that person's privacy interests, please feel free
to ask the instructor for guidance.
In addition, out of respect for each other, and in accordance with
federal guidelines such as FERPA, we will not share each other's
discussion posts or assignments without permission. As instructors, we will
ask you before sharing an assignment with a community sponser, for example.
Similarly, you should not share your fellow classmates' work without permission, and credit.
We also ask that you not share the ideas ideas presented in this class without credit.
While the class website is public, we ask that you do not take things out of context.
In addition, any tangible medium such as digital and physical documents are
protected by copyright law as embodied in title 17 of the United
States Code. These expressions include the work product of both:
(1) your student colleagues (e.g., any assignments published here
in the course environment or statements committed to text in a
discussion forum); and, (2) your instructor (e.g., the syllabus,
assignments, reading lists, and lectures). Within the constraints
of &quot;fair use,&quot; you may copy these copyrighted
expressions for your personal intellectual use in support of your
education here in the UW. Such fair use by you does not include
further distribution by any means of copying, performance or
presentation beyond the circle of your close acquaintances,
student colleagues in this class and your family. If you have any
questions regarding whether a use to which you wish to put one of
these expressions violates the creator's copyright interests,
please feel free to ask the instructor for guidance.
The essence of academic life revolves around respect not only for
the ideas of others, but also their rights to those ideas. It is
therefore essential that we take the utmost care that the ideas
(and the expressions of those ideas) of others always be handled
appropriately, and, where necessary, cited. When ideas or
materials of others are used (particularly in your creative
projects), they must be cited. The citation format is not that important -
as long as the source material can be located and the citation
verified, it's OK. In any situation, if you have a question,
please feel free to ask.
You must have the right to publish any of the images, videos,
text, or other media on your creative sites. This means you may
use:
- Media you have created or generated yourself (i.e. pictures you have created or taken yourself, text you have written yourself.)
- Images that are in the public domain (something from Wikipedia), or something with a creative commons license that allows for reuse without explicit permission of the owner.
- [Creative Commons Kiwi](https://creativecommons.org/about/videos/creative-commons-kiwi/) is a really informative video on Creative Commons licensing.
- Instructions on how to search for images that are fair use are [here](https://support.google.com/websearch/answer/29508?hl=en).
- You must cite any works that you use that you did not generate yourself (although technically you only need to cite things that are [CC Attribution](https://creativecommons.org/licenses/by/2.0/). A handy site for knowing how to add your citations is
[here](https://wiki.creativecommons.org/wiki/Best_practices_for_attribution).
assignments, reading lists, and lectures).
Within the constraints of <q>fair use,</q> you may copy these
copyrighted expressions for your personal intellectual use in support
of your education here in the UW. Such fair use by you does not
include further distribution by any means of copying, performance or
presentation beyond the circle of your close acquaintances, student
colleagues in this class and your family. If you have any questions
regarding whether a use to which you wish to put one of these
expressions violates the creator's copyright interests, please feel
free to ask the instructor for guidance.
# Collaboration Policies
> As a teacher, it’s not my goal in life to read a class’ worth of
> programs on a topic that all basically look the same. My goal is
> to assess whether you understand the material that we’ve taught in
> class enough, and you demonstrate that to me through the work you
> do. I can not assess that if you are turning in someone else’s
> work.
> - Dr. Tom Butler, Lakeside H.S.
Computer science education is odd in that we expect you to turn in
work that you do completely independently when in the
&quot;real world&quot; that’s not how it works at all. In the real
world, co-workers collaborate, bounce ideas off each other, they
look up parts of solutions on the internet. But in the &quot;real
world&quot; the people doing the work have years of experience,
they have proved themselves to their teachers, co-workers and
bosses to where they are at that moment and most importantly, they
know <strong>how to evaluate which of the solutions they are
receiving is an appropriate one to solve the task at hand.</strong>
As your instructor, I need to be able to evaluate
<strong>your</strong> work. Thus, unless otherwise specified all
work in this and other CS classes must be your own. We realize you
may look at other sources online to learn how to achieve new things, but we expect
you to synthesize this information and not copy it directly. You
should never copy (plagiarize) homework or code from another person
in this school (past or present) or that you find online directly
and submit it as your own work.
All programming assignments must be completed individually. However,
you **may discuss homework assignments with other students**
(i.e. provide advice on difficult concepts, *etc*) and are encouraged
to discuss class material, such as section exercises, lecture
material, readings, etc. Even better if this takes place on Ed where other students can benefit and we can guide you as to what is supportive and what crosses the line to too much sharing. You may also discuss Creative Projects with
other students, as long as the code you write is entirely your own,
and discussion with students on Creative Projects should never involve
details of how to code a solution. Specifically, you must abide by the
following:
- You **may not use code directly from any external sources** (including copying lecture/lab material directly into a programming assignments).
- You may not post your homework solutions on a publicly accessible (non-password-protected) web server or Git repository, during the course or after it has been completed. Please see the course website for acceptable ways to show your work to others.
- You may not look at or use prior solutions from any source.
- You must document substantive discussions (i.e. when a classmate's advice helped you with a difficult concept)
- You may not provide a classmate with actual code or with step-by-step instructions
In short: you should think of most assignments in this class as assessments and as such, complete them independently - unless otherwise told.
Under our policy, a student who gives inappropriate help is
equally guilty with one who receives it. Instead of providing such
help to a classmate, point them to other class resources such as
lecture examples, OH, or a TA. You must take reasonable steps
to ensure that your work is not copied by others, such as making
sure to log out or lock shared computers, not leaving printouts of
your code in public places, and not emailing code to other
students or posting it on the web or public forums.
We enforce our policies by running detection software during the
quarter over all programs, including ones from past quarters.
Please contact us if you are unsure whether a particular behavior
falls within our policy.
## If you make use of a source, or advice from a classmate
Some portions of our projects will have a creative aspect to them. On occasion students may wish to use portions of sample code that has been obtained on our course website or others. In order to avoid academic misconduct for a Creative portion of your projects you must:
- Ensure that substantive original work is submitted that can be evaluated by the course staff.
- Cite the ideas or materials of others that are used. The citation format is not that important - as long as the source material can be located and the citation verified (a url in a comment is generally fine), it's OK.
- Clearly indicate (e.g. with comments) which portions of your code are completely original and which are used or modified from external sources, if any code is used that builds off of/is inspired by external sources (e.g. adaption of an example exercise, online tutorial you find). We will only grade your original work. Note that solely changing identifier names or rearranging other source material is not considered your original work - see the examples of appropriate use below for details.
A good analogy to this is if you were writing a history paper: You can use quotes in your paper as long as you give attribution to the sources of the quote, but you can not write a history paper out of the quotes of others (particularly with no citations).
Some examples of appropriate use:
- A student closely follows a tutorial to understand a new concept in Android Development (e.g. animations). The student cites the tutorial they used in the file header then substantially modifies the tutorial code to include what is specified for the creative portion of the assignment, documenting which portions of the code are their own so TAs know which portions to grade (and to determine whether the material cited as being learned from the tutorial is sufficiently adapted to be considered the student’s own work).
- A student is having difficulty styling their website. They look for a solution and find one on a site such as Stack Overflow. The student uses the code they find in their solution, documents that small piece of code was not their own with a comment that includes where it was found. The TAs will not use that portion of the code in grading.
Students with questions about any specific situation should ask the instructor for clarification.
# Appropriateness
# Privacy
Recall that one of our course policies is to engender an
inclusive environment. As such it is important that you are thoughtful about
what you say or write. Please make sure that
images and text you are using are <q>school appropriate</q> and follow
the guidelines of expected behavior. If you have any questions,
please do not hesitate to ask the TA or your instructor. Inappropriate work
submitted may be ineligible for credit on that assignment.
To support an academic environment of rigorous discussion and open
expression of personal thoughts and feelings, we, as members of
the academic community, must be committed to the inviolate right
of privacy of our student and instructor colleagues. As a result,
we must forego sharing personally identifiable information about
any member of our community including information about the ideas
they express, their families, lifestyles and their political and
social affiliations. If you have any questions regarding whether a
disclosure you wish to make regarding anyone in this course or in
the university community violates that person's privacy interests,
please feel free to ask the instructor for guidance.
Knowingly violating any of these principles of academic conduct,
privacy or copyright may result in University disciplinary action
under the Student Code of Conduct.
......@@ -144,16 +144,14 @@ table tbody td {
table thead {
background: $dark;
color: $light;
border-bottom: 4px solid $dark;
border-left: none;
border-right: none;
}
table thead th {
table thead tr {
font-size: 0.7em;
font-weight: bold;
color: $light;
border-left: 2px solid $dark;
}
table thead th:first-child {
border-left: none;
......
......@@ -307,6 +307,7 @@ table.schedule tfoot td {
.page-header {
background-image: linear-gradient(-120deg, $gradient-light, $gradient-dark, $gradient-dark);
padding-top: 1rem;
padding-bottom: 1rem;
}
.project-tagline {
......
......@@ -13,7 +13,7 @@ will update this page as rules/regulations change.
# Masking
Masking is currently optional, but ``recommended indoors'' according to the [UW face covering policy](https://www.ehs.washington.edu/covid-19-prevention-and-response/face-covering-policy).
Masking is currently optional, but ``strongly recommended indoors'' according to the [UW face covering policy](https://www.ehs.washington.edu/covid-19-prevention-and-response/face-covering-policy).
In this class, people in the room include those who are high risk with respect to COVID and people who live with vulnerable family members. To help support the needs of these individuals, masks will be available in the classroom as you enter, and if you would like to mask, you are welcome to make use of them.
......
......@@ -62,6 +62,8 @@ Taking a class is a big commitment, and you will work hard in this class. So we
The only requirement for this class is that you are comfortable programming and picking up new languages and tools that you have not been exposed to before. You will have some control over this, however, basic web skills are likely to be useful. The primary programming project in this class is one you design yourself.
In addition, please familiarize yourself with the course [academic conduct]({{site.baseurl}}/academic-conduct.html) policy. We will grade on the assumption that you have read and understood it.
# Course Structure
Many of the goals in this class center around learning by doing. This
......
......@@ -323,7 +323,7 @@ Discussion of Disability Dongle Paper
{: .activity} 7:00-7:40 Skill Goal: Various Techniques to Make Visualizations Accessible
: **In class Exercise**
- Implement a visualization using [VoxLens](https://github.com/athersharif/voxlens) following the documentation and examples on the GitHub repository.
<!-- - Implement a visualization using [VoxLens](https://github.com/athersharif/voxlens) following the documentation and examples on the GitHub repository. -->
- Use VoxLens from to test different modes of making visualizations accessible (Summary, Sonification, Question-and-Answer) on different visualization types.
- Identify the utility of each mode with each visualization type–when is one mode more suitable than the other and why.
- Identify accessibility measures for demographics other than screen-reader users.
......
......@@ -23,6 +23,7 @@ class:
## This is an important reminder
## Make sure zoom is running and recording!!!
## Make sure captioning is turned on
---
layout:false
......
......@@ -21,21 +21,31 @@ class:
## This is an important reminder
## Make sure zoom is running and recording!!!
## Make sure captioning is turned on
---
# COVID-19 Visualizations
![:img Visualizations of COVID-19 cases taken from different websites including Seattle Times and King County’s official website,80%, width](img/data-equity/covid-19-viz.png)
---
[//]: # (Outline Slide)
# Learning Goals for Today
- What are the commonly-used techniques to make data visualizations accessible?
# Some questions to consider about information access
Who can access visualizations that other people have created?
- What are the pros and cons of these techniques?
Who can create accessible information?
- What are some of the nuances in making data visualizations accessible?
How do we support the full data science lifecycle?
---
# COVID-19 Visualizations
# Who can Access Visualizations?
Due to the inaccessibility of online data visualizations, screen-reader users answered questions **62%** less accurately and spent **211%** more time than non-screen-reader users *(Sharif, 2021)*.
![:img Visualizations of COVID-19 cases taken from different websites including Seattle Times and King County’s official website,100%, width](img/data-equity/covid-19-viz.png)
![:img Two different graphs comprising bar chart of Accuracy of Extracted Information per Visualization Library and interaction plot of Interaction Times per Visualization Library per participant group.,80%, width](img/data-equity/sru-viz-results.png)
---
# How Can We Make These Visualization Accessible?
......@@ -47,13 +57,13 @@ class:
---
# Commonly-Used Techniques
- Summary ("Alt-Text")
Summary ("Alt-Text")
- Sonification (Audio Graphs)
Sonification (Audio Graphs)
- Tables
Tables
- Braille Prinouts/3-D Printing
Braille Prinouts/3-D Printing
---
# Summary ("Alt-Text")
......@@ -67,15 +77,19 @@ Alt-text is an HTML attribute that can provide a summary of the visualization an
---
# Sonification (Audio Graphs)
Sonification (or audio graphs) enable listeners to interpret data trends by mapping data to a musical scale.
[Example](https://www.youtube.com/watch?v=gssjuSnMR60)
.left-column40[
**Benefits**
- Can help users to understand overall trends
- Already integrated into commerical products (e.g., Apple)
]
.right-column60[
![:youtube Video of a graph being sonified, gssjuSnMR60]
]
---
# Tables
......@@ -98,34 +112,35 @@ Tactile graphics and 3-D printed graphs are raised surfaces that enable blind an
- Enables users to interact with the visualization without using a screen reader
---
# Equality vs. Equity
- What does **equal** access mean?
- What does **equitable** access mean?
- Are these techniques equitable?
# Holistic vs. Drilled-Down Exploration
---
# State of The World
Screen-reader users perform a holistic exploration first and then drill-down into the data *(Sharif, 2021)*.
Due to the inaccessibility of online data visualizations, screen-reader users answered questions **62%** less accurately and spent **211%** more time than non-screen-reader users *(Sharif, 2021)*.
.left-column50[
- Summary ("Alt-Text")
- Sonification (Audio Graphs)
- Tables
- Braille Prinouts/3-D Printing
]
.right-column50[
![:img Two different graphs comprising bar chart of Accuracy of Extracted Information per Visualization Library and interaction plot of Interaction Times per Visualization Library per participant group.,100%, width](img/data-equity/sru-viz-results.png)
- Which of these techniques enable **holistic** exploration?
- Which of these techniques enable **drilled-down** exploration?
]
---
# Holistic vs. Drilled-Down Exploration
# Supporting Interaction
Screen-reader users perform a holistic exploration first and then drill-down into the data *(Sharif, 2021)*.
- Which of these techniques enable **holistic** exploration?
Highly relevant even when just *using* a visulazation
- When interacting with static visualizations, sighted people intuitively construct complex visual queries [Ware, 2019].
- Interactive charts afford selection, filtering & more [Yi, IEEE TOVCG, ’07]
- Which of these techniques enable **drilled-down** exploration?
Supporting this requires going past verbal descriptions and tactile graphics
---
# VoxLens
# [VoxLens](https://github.com/athersharif/voxlens).
VoxLens is an open-source JavaScript plug-in that improves the accessibility of online data visualizations for screen-reader users using a multi-modal approach, requiring only a single line of code for integration into online data visualizations.
[VoxLens](https://github.com/athersharif/voxlens) is an open-source JavaScript plug-in that improves the accessibility of online data visualizations for screen-reader users using a multi-modal approach, requiring only a single line of code for integration into online data visualizations.
VoxLens offers three modes:
......@@ -135,6 +150,11 @@ VoxLens offers three modes:
.center[https://github.com/athersharif/voxlens]
---
# VoxLens
![:img Quering Voxlens. First query is "What's the maximum data point in the graph?" with answer "I understood you're looking for maximum. Maximum value of Total Crime for Crime Neighborhood is 48900 belonging to Downtown Commercial." Second query is "What about average and variance?" with answer "I heard you asking about the average and variance. Average of Total Crime for Crime Neighborhood is 8870. Variance of Total Crime for Crime Neighborhood is 78800000" ,50%, width](img/data-equity/voxlens-interactive.png)
---
# Evaluation of VoxLens
......@@ -143,27 +163,149 @@ VoxLens improved the accuracy of screen-reader users by **122%** and interaction
![:img Two different graphs comprising bar chart of Accuracy of Extracted Information per Visualization Library and interaction plot of Interaction Times per Visualization Library per participant group.,100%, width](img/data-equity/voxlens-viz-results.png)
---
# Class Activity
[//]: # (Outline Slide)
**Activitiy**
- Break into small groups.
- Implement a visualization using [VoxLens](https://github.com/athersharif/voxlens).
- Use VoxLens from to test different modes of making visualizations accessible (Summary, Sonification, Question-and-Answer).
# Some questions to consider about information access
**Goals**
- Identify the utility of each mode with each visualization type–when is one mode more suitable than the other and why.
- Identify accessibility measures for users other than screen-reader users.
- Create a presentation (2-3 slides) to share findings
Who can access visualizations that other people have created?
.center[https://github.com/athersharif/voxlens]
**Who can create accessible information?**
How do we support the full data science lifecycle?
---
# *Who* Can Creating Accessible Visualizations
Can BLV people create new information
- Are they included in the employment opportunities and learnings associated with data science?
- Can they access information when someone didn’t already decide it needs to be accessible
- Maps and other individualized needs
- Live data exploration
---
# Non Programmers
.left-column40[
## Maptimizer
![:img A 3d printed tactile map of the UW campus showing the fountain and the grassy area around it,100%, width](img/data-equity/maptimizer.jpg)
(Hofmann, CHI 2022)
]
.right-column60[
## Generative Design
*Generative design* is a common approach to support end users (used a lot for fabrication)
Lets each user specify their information priorities and preferred representations in familiar terms through a simple GUI
]
---
# What About Direct Authoring?
We should also support more advanced creation
- In Kim et al.’s survey of 56 accessible visualization papers published between 1999 and 2020 only two papers supported interactive authoring of charts by BVI users [Kim, CGF ‘21]
Equivalent experiences is an important access goal
- Same information conveyed
- Same *interactive experience*
---
# PSST
Based on RxJS: everything is a stream
.left-column40[
<div class="mermaid">
graph TD
LS[Sensor] --> DS[DataSink]
OE[OutputEngine] --> DS
DS -->|Sensor values| NH[Handler]
NH -->|converts sensor values to appropriate format| O[Output]
</div>
]
---
# PSST Example
Sonifying a light sensor
.left-column40[
<div class="mermaid">
graph TD
LS[Light Sensor] --> DS[DataSink]
OE[OutputEngine] --> DS
DS -->|Light sensor values| NH[NoteHandler]
NH -->|converted light sensor values to audible range| O[NoteOutput]
</div>
]
.right-column60[
![:youtube A sound can be heard with the pitch rising and lowering as a light moves closer and further from the circuit board, aSe-QkSl12c]
]
<!-- --- -->
<!-- # PSST -->
<!-- PSST allows BVI developers to make their own data visualizations -->
<!-- <div class="mermaid"> -->
<!-- sequenceDiagram -->
<!-- Lightsensor->>+DataSink: New Value as Datum -->
<!-- OutputEngine->>+DataSink: Status is "Playing" -->
<!-- DataSink-\->>+NoteHandler: Status, Datum -->
<!-- NoteHandler->>+NoteOutput: Status, Datum adjusted using Mel Scale -->
<!-- NoteOutput->>+User: Plays note if status is "Playing" -->
<!-- </div> -->
---
# Code or GUI Configuration
.column[
GUI automatically created from spec
User can select, parameterize and attach outputs to streaming data
]
.column[
![:youtube A user configuring a sonification using the accessible PSST GUI while a screenreader plays a description, xl70zqfW2H0]
]
--
.column[
.quote[I’ve seen these sonifications for decades . . . but I’ve never seen anything nice in a dashboard like this that lets you add and remove multiple ones and tweak the parameters of each.]
]
---
[//]: # (Outline Slide)
# Learning Goals for Today
- What are the commonly-used techniques to make data visualizations accessible?
# Some questions to consider about information access
Who can access visualizations that other people have created?
Who can create accessible information?
**How do we support the full data science lifecycle?**
- Data collection
- Data cleaning
- Modeling and Analysis
- Representation
- What are the pros and cons of these techniques?
---
# Data Science Accessibility
Very much still a wide open area, at least for BLV people
- Data collection -- not particularly supported (even coding tools very inaccessible)
- Data cleaning -- not studied to my knowledge
- Modeling and Analysis -- Common tools like Jupyter Notebooks wholly inaccessible. Venkatesh (first author of PSST) is starting to work on this
- Representation -- Accessible visualization & Data physicalization are the big opportunities
---
# One last PSST Video
... and then on to our guest!
- What are some of the nuances in making data visualizations accessible?
![:youtube A user turns a handle on a music box. A laser cut strip containing data moves through producing notes, EKaaP0TKQog]
......@@ -21,6 +21,7 @@ class:
## This is an important reminder
## Make sure zoom is running and recording!!!
## Make sure captioning is turned on
---
[//]: # (Outline Slide)
......@@ -54,10 +55,94 @@ But *how*?
- How does Machine Learning solve a problem?
---
# Components of Machine Learning
# Helping Computers Learn Patterns
- Collect data (and lots and lots of it!)
.left-column50[
![:fa bed, fa-7x]
]
.right-column50[
## How might you recognize sleep?
- Can you come up with a yes/no question or a set of categories or simple description of sleep?
- Sleep quality?
- Sleep start/end?
- What data would you learn from?
- How might you need to take disabilites into account?
]
???
(sleep quality? length?...)
How to interpret sensors?
---
# How do we program this?
Old Approach: Create software by hand
- Use libraries (like JQuery) and frameworks
- Create content, do layout, code up functionality
- Deterministic (code does what you tell it to)
New Approach: Collect data and train algorithms
- Will still do the above, but will also have some functionality based
on ML
- *Collect lots of examples and train a ML algorithm*
- *Statistical way of thinking*
---
# Shift in Approaches
.left-column50[
## Old style of app design
<div class="mermaid">
graph TD
I(Input) --Explicit Interaction--> A(Application)
A --> Act(Action)
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
class U,C,A,I,S,E,Act,Act2 normal
</div>
]
--
count: false
.right-column50[
## New style of app design
<div class="mermaid">
graph TD
U(User) --Implicit Sensing--> C(Application)
S(System) --Implicit Sensing--> C
E(Environment) --Implicit Sensing--> C
C --> Act2(Action)
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
class U,C,A,I,S,E,Act,Act2 normal
</div>
]
---
# This is *Machine Intelligence*
Often used to process sensor data
Goal is to develop systems that can improve
performance with more experience
- Can use "example data" as "experience"
- Uses these examples to discern patterns
- And to make predictions
Not really intelligent, just my word for Machine Learning, AI, Neural Programming, etc etc
---
# Components of Machine Intelligence
- **Collect data (and lots and lots of it!)**
- Discern patterns
......@@ -73,49 +158,370 @@ But *how*?
- Who do we collect data from?
---
# "Goodness" of Data
# Problems with Data
- System timeouts that are trained on movement speeds of <q>typical</q> people
- Biometrics that cannot function on a person who isn't still for long enough
- Inferencing about people that doesn't account for height; stamina; range of motion; or AT use (e.g. wheelchairs)
When groups are historically
marginalized and underrepresented, this is
.quote[imprinted in the data that shapes AI
systems... Those who have borne discrimination in the past are most at risk of harm from
biased and exclusionary AI in the present. (Whittaker, 2019)]
--
This can cascade -- e.g. measurement bias can exacerbate bias downstream. For example, facial mobility, emotion expression, and facial structure impact detection and identification of people; body motion and shape impact activity detection; etc.
---
# How might we address bias/fairness in data sets
We need to know it is there (Aggregate metrics can hide performance problems in under-represented groups)
We need to be careful not to eliminate, or reduce the influence, of outliers if that erases disabled people from the data because of the heterogeneity of disability data.
---
# Approaches to measuring fairness
We may need to rethink <q>fairness</q> in terms of individual rather than group outcomes, and define metrics that capture a range of concerns
- Movement speed might favor a wheelchair user
- Exercise variety might favor people who do not have chronic illness
- Exertion time might covers a wide variety of different types of people.
Defining such unbiased metrics requires careful thought and domain knowledge, and scientific research will be essential to defining appropriate procedures for this.
---
# Small Group Discussion [Post on Ed]({{site.discussion}}2514887)
Who might be excluded in the data set you found?
How was fairness measured in the data set you found, if it was discussed?
How would you go about testing for fairness in that data?
---
# Best Practices For Data Fairness
How do we motivate and ethically compensate disabled people to give their data?
What should we communicate at data collection time?
Is the data collection infrastructure accessible? Does it protect sensitive information about participants adequately given the heterogeneous nature of disability?
Does the meta data collected oversimplify disability? Who is labeling the data and do the have biases affecting labeling?
- Whittaker (2019) discusses the example of clickworkers who label people
as disabled <q>based on a hunch</q>.
---
# Components of Machine Intelligence
- Collect data (and lots and lots of it!)
- **Discern patterns**
- Make predictions
---
# Two main approaches
![:fa eye] *Supervised learning* (we have lots of examples of what should be
predicted)
![:fa eye-slash] *Unsupervised learning* (e.g. clustering into groups and inferring what
they are about)
![:fa low-vision] Can combine these (semi-supervised)
![:fa history] Can learn over time or train up front
---
# Our Focus: Supervised Learning
![:fa eye] *Supervised learning* (we have lots of examples of what should be
predicted)
![:fa eye-slash] *Unsupervised learning* (e.g. clustering into groups and inferring what
they are about)
![:fa low-vision] Can combine these (semi-supervised)
![:fa history] Can learn over time or train up front
---
# Supervised Learning
.left-column50[
## Training Process
<div class="mermaid">
graph TD
L(Label) --> MI(Training Algorithm)
D(Input Data) -- Extract Features--> MI
MI --> C(Symbolic Predictor)
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
class D,U,C,A,I,S,E,Act,Act2 normal
</div>
]
.right-column50[
## Extracting Features
Symbolic requires feature engineering (humans deciding how to *summarize* data using features. Tends to be more *interpretable* (you can figure out why they make predictions)
]
---
# Supervised Learning
.left-column50[
## Training Process
<div class="mermaid">
graph TD
L(Label) --> MI(Training Algorithm)
D(Input Data) --> MI
MI --> C(Neural Predictor)
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
class D,U,C,A,I,S,E,Act,Act2 normal
</div>
]
.right-column50[
## Designing Networks
Neural approaches (e.g. ChatGPT) use massive amounts of data to train a network according to base principales. Designing the right network is critical. Cannot be sure *why* they make the predictions they do.
]
---
# Supervised Learning
.left-column50[
## Training Process
<div class="mermaid">
graph TD
L(Label) --> MI(Training Algorithm)
D(Input Data) --> MI
MI --> C(Symbolic/Neural Predictor)
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
Let's consider this example: **We wish to train a ML model to recognize if a given picture is that of a nurse.**
class D,U,C,A,I,S,E,Act,Act2 normal
</div>
]
.right-column50[
## Prediction Process
What steps do we take?
<div class="mermaid">
graph TD
Is the data "good"?
D(Input Data) --> C(Symbolic/Neural Predictor)
C --> P(Prediction)
classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
class P,D,U,C,A,I,S,E,Act,Act2 normal
</div>
]
What other examples can we think of?
---
# "Goodness" of Data
# How do we Evaluate Predictors?
Compare to Prior probabilities
- Probability before any observations (ie just guessing)
- Ex. ML classifier to guess if an animal is a cat or a ferret based on the ear location
- Assume all pointy eared fuzzy creatures are cats (some percentage will be right)
Compare to simplistic algorithms
- Ex. Classifying cats vs ferrets based on size
- Your model needs to do better than these too
Surprising how often this doesn't happen in published work/before deployment
???
We did this to study gender's impact on academic authorship; doctors reviews
---
# Adding Nuance
.left-column50[
## <q>Confusion Matrix</q>
![:img Confusion matrix of a machine learning model,100%, width](img/data-equity/ml-faulty.png)
]
.right-column50[
Don't just measure accuracy (percent correct)
Lots of other metrics based on false positives and negatives
- Precision = TP / (TP+FP) Intuition: Of the positive items, how many right?
- Recall = TP / (TP+FN) Intuition: Of all things that should have been positive, how many actually labeled correctly?
- ... Many More
]
---
# Disability Bias in Machine Learning
# Using Proper Methods
Data sets used to train biometric systems are biased: they rarely if ever have a comprehensive
representation of the range of people they might encounter. *(Mankoff, 2021)*
.left-column50[
**Symbolic Methods Can Easily Overfit**
- Human bias
When your ML model is too specific for data you have, it might not generalize well
- Machine bias
Best test is a data set you haven't seen before
- Underrepresentation
![:img overfitting is illustrated as a line snacking between data points to minimize error instead of smoothly rising among them , 80%,width](img/data-equity/overfitting.png)
- Measurement Errors
]
.right-column50[
**Neural Methods Can Have Hidden Biases**
![:img A headline from the Verge stating that Twitter taught Microsoft's AI Chatbot to be a racist asshole in less than a day, 80%,width](img/data-equity/racist-chatbot.png)
]
---
# Minimizing Bias
# Disability Biases to Watch Out For
- Appropriate representation
Norms are baked deeply into algorithms which are designed to learn about the most common cases. As human judgment is increasingly replaced by biometrics, *norms* become more strictly enforced.
- Do outliers face higher error rates?
- Do they disproportionately represent and misrepresent people with disability?
- How does this impact allocation of resources?
- Large training data
---
# How does norming harm people with disabilities?
- Validating the models
Machine intelligence already being used to track allocation of assistive technologies, from CPAP machines for people with sleep apnea (Araujo 2018) to prosthetic legs (as described by Jullian Wiese in
Granta and uncovered in Whittaker et al 2019), deciding who is <q>compliant enough</q> to deserve them.
- Montoring the performance of the models
Technology may also fail to recognize that a disabled person is even present (Kane, 2020), thus <q>demarcating what it means to be a legible human and
whose bodies, actions, and lives fall outside... [and] remapping and calcifying the boundaries
of inclusion and marginalization</q> (Whittaker, 2019).
---
# Discussion
# How does norming harm people with disabilities?
- Identify biases in existing ML-based tools/platforms
Many biometric systems gatekeep access based on either individual identity, identity as a human, or class of human, such as <q>old enough to buy cigarettes.</q>
Examples:
- a participant having to falsify data because <q>some apps [don’t allow] my height/weight combo for my age.</q> (Kane (2020))
- a person who must ask a stranger to ‘forge’ a signature at the grocery store <q>.. because I can’t reach [the tablet]</q> (Kane (2020))
- at work, activity tracking may define <q>success</q> in terms that exclude disabled workers. (may also increase the likelihood of work-related disability, by forcing workers to work at maximal efficiency)
- Identify ways to make ML models more inclusive in your area of expertise
---
# Components of Machine Intelligence
- Collect data (and lots and lots of it!)
- Discern patterns
- **Make predictions**
---
# Concerns at Prediction Time
Denial of insurance and medical care, or threaten employment (Whittaker,
2019, p. 21).
- HireVue, an AI based video
interviewing company has a patent on file to detect disability (Larsen, 2018).
- This is illegal under the ADA, which
- forbids asking about disability
status in a hiring process (42 U.S.C. § 12112(a))
- forbids <q>using qualification
standards, employment tests or other selection criteria that screen out or tend to screen out
an individual with a disability</q> (42 U.S.C. § 12112(b)(6)).
---
# Concerns at Prediction Time
Denial of insurance and medical care, or threaten employment
Disability identification
- Examples: detect Parkinsons from gait (Das, 2012), and mouse movement (Youngmann,
2019); detecting autism from home videos (Leblanc, 2020).
- What are the ethics of doing this without consent?
- Many of these algorithms encode medical model biases
Relatedly, failure to identify disability
- Legally under the ADA, if you are treated as disabled, you are disabled. Yet biometrics cannot detect how people are treated.
---
# Concerns at Prediction Time
Denial of insurance and medical care, or threaten employment
Disability Identification / Failure to Identify
Apps that Harm
- Example: Training behaviors in <q>support</q> of autistic individuals without regard to debates about agency and independence of the target audience [Demo, 2017];
- As with regular accessibility apps, AI based aps can harm, be disability dongles, etc
- As with regular apps, AI based apps may not be accessible
---
# Concerns at Prediction Time
![:img Three news headlines-- On Orbitz Mac Users Steered to Pricier Hotels; Google's algorithm shows prestigious job ads to men but not to women; Racial bias alleged in Google's add results, 60%,width](img/data-equity/bias.png)
---
# Concerns at Prediction Time
Denial of insurance and medical care, or threaten employment
Disability Identification / Failure to Identify
Apps that Harm
AI with Baked in Biases
- Consequences of biased data and lack of control over training results more nuanced than just accuracy (as with headlines we just read)
- Privacy can also be a concern.
- For rare conditions, an algorithm may learn
to recognize the disability, rather than the individual, reducing
security when used for access control, allowing multiple people with
similar impairments to access the same data.
---
# Concerns at Prediction Time
Denial of insurance and medical care, or threaten employment
Disability Identification / Failure to Identify
Apps that Harm
AI with Baked in Biases
Transparency and Accountability
- Power differences between builders and users
- Representation of disabled people among builders
- Algorithms that are not *interpretable* or *correctable*
- Users of algorithms whose use them to enforce larger societal harms
---
# Small Group Discussion [Post on Ed]({{site.discussion}}2515387)
Revisit the data set you chose
Do you know what sort of predictions it was used for if any?
What possible harms could be done with those predictions?
Reminder of our list
- Denial of insurance and medical care, or threaten employment
- Disability Identification / Failure to Identify
- Apps that Harm
- AI with Baked in Biases
- Transparency and Accountability
---
# End of Deck
slides/img/data-equity/bias.png

458 KiB

slides/img/data-equity/gma.png

253 KiB

slides/img/data-equity/maptimizer.jpg

121 KiB

0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment