A lot of changes to the main website as well as class materials for this week.

_config.yml

@@ -73,8 +73,7 @@ sass:
 kramdown:
   toc_levels: '1'
 
-mermaid:
-  src: 'https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js'
+mermaid: 'https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js'
 
 remarkConfig:
   # possible highlight options:

_config_production.yml

@@ -69,8 +69,7 @@ sass:
 kramdown:
   toc_levels: '1'
 
-mermaid:
-  src: 'https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js'
+mermaid: 'https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js'
 
 remarkConfig:
   # possible highlight options:

_data/navigation.yml

@@ -10,5 +10,7 @@ nav_list:
   url: canvas
 - title: Ed
   url: discussion
-- title: COVID-19 Safety
+- title: COVID-19
   url: /docs/covid/index.html
+- title: Academic Conduct
+  url: /academic-conduct.html
\ No newline at end of file

_includes/header.html

 <header class="page-header" role="banner">
   {% include navigation.html %}
-  <br>  <br>  <br>  <br>  <br>
+  <br>  <br>  <br>  <br>
   <h1 class="project-name">{{ site.title | default: site.github.repository_name }}</h1>
   <h2 class="project-tagline">{{ site.description | default: site.github.project_tagline }}</h2>
 
-
 <script type="text/javascript">
   var menubar = new Menubar(document.getElementById('menubar1'));
   menubar.init();

_layouts/assignment.html

@@ -55,7 +55,7 @@
       {% include footer.html %}
     </main>
 
-    <script type="text/javascript" src="{{'assets/js/mermaid.8.8.2.js' | relative_url}}"></script>
+    <script type="text/javascript" src="{{site.mermaid}}"></script>
   <link rel="stylesheet" href="{{ '/assets/css/mermaid.css' | relative_url }}">
   <link rel="stylesheet" href="{{ '/assets/css/assignment.css' | relative_url }}">
   <script>

_layouts/default.html

@@ -24,7 +24,7 @@
 
   <script src="{{ '/assets/js/bootstrap.min.js' | relative_url}}"></script>
 
-    <script type="text/javascript" src="{{'assets/js/mermaid.8.8.2.js' | relative_url}}"></script>
+    <script type="text/javascript" src="{{site.mermaid}}"></script>
   <link rel="stylesheet" href="{{ '/assets/css/mermaid.css' | relative_url }}">
   <script>
         var config = {

_layouts/presentation.html

@@ -45,7 +45,7 @@
     <script type="text/javascript" src="{{'assets/js/mathjax-sre.js' | relative_url}}"></script>
     <script type="text/javascript" src="{{'assets/js/html2canvas.min.js' | relative_url}}"></script>
     <script type="text/javascript" src="{{'assets/js/presentation-export.js' | relative_url}}"></script>
-    <script type="text/javascript" src="{{'assets/js/mermaid.8.8.2.js' | relative_url}}"></script>
+    <script type="text/javascript" src="{{site.mermaid}}"></script>
 
     <script type="text/javascript">
       var slideshow = remark.create({

_layouts/presentation_production.html

@@ -45,7 +45,7 @@
     <script type="text/javascript" src="{{'assets/js/mathjax-sre.js' | relative_url}}"></script>
     <script type="text/javascript" src="{{'assets/js/html2canvas.min.js' | relative_url}}"></script>
     <script type="text/javascript" src="{{'assets/js/presentation-export.js' | relative_url}}"></script>
-    <script type="text/javascript" src="{{'assets/js/mermaid.8.8.2.js' | relative_url}}"></script>
+    <script type="text/javascript" src="{{site.mermaid}}"></script>
 
     <script type="text/javascript">
       var slideshow = remark.create({

academic-conduct.md

@@ -3,22 +3,21 @@ layout: default
 (no)warning: old
 ---
 
-
----
-
 Integrity is a crucial part of your character and is essential for
 a successful career. We expect you to demonstrate integrity in
-CSE 154 and elsewhere.
-
-The Paul G Allen School has an entire page on
-[Academic Misconduct](https://www.cs.washington.edu/academics/misconduct)
-within the context of Computer Science, and the University of
-Washington has an entire page on how
-[Academic Misconduct](https://www.washington.edu/cssc/for-students/academic-misconduct/)
-[Community Standards and Student Conduct Page](https://www.washington.edu/cssc/).
-Please acquaint yourself with both of those pages, and in
-particular how academic misconduct will be reported to the
-University.
+this class and elsewhere.
+
+The Paul G Allen School has an [Academic
+Misconduct](https://www.cs.washington.edu/academics/misconduct)
+policy, and the University of Washington an [Academic
+Misconduct](https://www.washington.edu/cssc/for-students/academic-misconduct/)
+and [Community Standards and Student Conduct
+Page](https://www.washington.edu/cssc/) policy. Please acquaint
+yourself with those pages, and in particular how academic misconduct
+will be reported to the University. Knowingly violating any of these principles of academic conduct,
+privacy or copyright may result in University disciplinary action
+under the Student Code of Conduct.
+
 
 Your academic conduct in this course is evaluated in at least the
 four areas described in detail below.
@@ -26,160 +25,110 @@ four areas described in detail below.
 * TOC
 {:toc}
 
-# Honesty in Communications
+# Honesty and Respect in Communications
 
 Individuals are expected to be honest and forthcoming
-in communications with TAs and the instructor.
+in communications with TAs and the instructor. 
+
+In addition, individuals are expected to show respect for the
+intellectual contributions of others through citation. The essence of
+academic life revolves around respect not only for the ideas of
+others, but also their rights to those ideas. It is therefore
+essential that we take the utmost care that the ideas (and the
+expressions of those ideas) of others always be handled appropriately,
+and, where necessary, cited. This is an issue of [Citational
+Justice](nature.com/articles/d41586-022-00793-1), and a core value of
+this course. 
+
+When ideas or materials of others are
+used (particularly in your creative projects), they must be cited. The
+citation format is not that important - as long as the source material
+can be located and the citation verified, it's OK. In any situation,
+if you have a question, please feel free to ask. Here are some examples of how you might use (and cite) different types of content:
+
+- Media you have created or generated yourself (i.e. pictures you have created or taken yourself, text you have written yourself) do generally not require citation. However if you have published them (on a blog, in an article etc), they may belong to the publisher and require citation.
+-  Images that are in the public domain (something from Wikipedia), or with a creative commons license that allows for reuse without explicit permission of the owner, require citation based on their license. Instructions on how to search for images that are fair use are [here](https://support.google.com/websearch/answer/29508?hl=en). [Creative Commons Kiwi](https://creativecommons.org/about/videos/creative-commons-kiwi/) is a really informative video on Creative Commons licensing; and here are [best practices](https://wiki.creativecommons.org/wiki/Best_practices_for_attribution) for citing Creative Commons works.
+- If you are writing some text (a paper, analysis, etc), you can use quotes as long as you give attribution to the sources of the quote. You can not write an entire document out of the quotes of others, or include copied text with no citations. The citation format is not that important - as long as the source material can be located and the citation verified (a url in a comment is generally fine), it's OK. 
+- If you are writing code, clearly indicate (e.g. with comments) which portions of your code are completely original and which are used or modified from external sources, if any code is used that builds off of/is inspired by external sources (e.g. adaption of an example exercise, online tutorial you find). Note that solely changing identifier names or rearranging other source material is not considered your original work - see the examples of appropriate use below for details.
 
-# Application Content
-
-## School Appropriateness of Content
+Some examples of appropriate use:
+- A student finds a blog post explaining why and how to address WCAG guidelines. They rewrite much of it in their own words, but for a few sections that are particularly clear, they copy the information, put it in quotations, and name the source and provide a link to it right next to the quote.
+- A student closely follows a tutorial to understand a new concept in  Android Development (e.g.  animations). The student cites the tutorial they used in the file header then substantially modifies the tutorial code to include what is specified for the creative portion of the assignment, documenting which portions of the code are their own so TAs know which portions to grade (and to determine whether the material cited as being learned from the tutorial is sufficiently adapted to be considered the student’s own work).
+- A student is having difficulty generating audio from text for an accessibility feature of an app they are building. They look for a solution and find an app very similar to the one they intend to build. They fork it, and modify it for their final project, and documents this with a comment that includes where it was found. When grading the app, the instructional staff may weight features that were provided by the original source  less than features that the student added themselves.
 
-Recall that one of our course policies is to engender an
-inclusive environment. As such it is important that you are thoughtful about
-what you choose to post on your page. Please make sure that the
-images and text you are using are “school appropriate” and follow
-the guidelines of expected behavior. If you have any questions,
-please do not hesitate to ask a TA or your instructor. Inappropriate work
-submitted may be ineligible for credit on that assignment.
+Students with questions about any specific situation should ask the instructor for clarification.
 
+# Collaboration Policies
 
-## Copyright and Citations
+In this class, are encouraged to  discuss class material, including
+assignments, lecture material and readings with your classmates. Even
+better if this takes place on Ed where other students can benefit and
+we can guide you as to what is supportive and what crosses the line to
+too much sharing.
+
+Some assignments are individual. Even when as assignment is
+individual, you **may discuss homework assignments with other
+students** (i.e. provide advice, brainstorm) as long your writing
+and/or implementation is entirely your own, and you document what you
+do. You may also look at other sources online to learn how to achieve
+new things, but we expect you to document this, and it may impact the
+credit you get for your work. You should never copy (plagiarize) from
+another person in this school (past or present) or from material that
+you find online directly and submit it as your own work.
+
+To facilitate this, and to be very clear 
+- You **may not use code or writing directly from any external
+  sources** (including copying lecture/lab material directly into an
+  assignment) without appropriately crediting the source as described above in <q>Honesty and Respect in Communications</q>
+- You must credit a classmate when their advice had a significant intellectual impact on what you did
+
+# Privacy and Fair Use
 
-All of the expressions of ideas in this class that are fixed in
-any tangible medium such as digital and physical documents are
+To support an academic environment of rigorous discussion and open
+expression of personal thoughts and feelings, we, as members of the
+academic community, must be committed to the inviolate right of
+privacy of our student and instructor colleagues. As a result, we must
+forego sharing personally identifiable information about any member of
+our community including information about the ideas they express,
+their families, lifestyles and their political and social
+affiliations. If you have any questions regarding whether a disclosure
+you wish to make regarding anyone in this course or in the university
+community violates that person's privacy interests, please feel free
+to ask the instructor for guidance.
+
+In addition, out of respect for each other, and in accordance with
+federal guidelines such as FERPA, we will not share each other's
+discussion posts or assignments without permission. As instructors, we will 
+ask you before sharing an assignment with a community sponser, for example.
+Similarly, you should not share your fellow classmates' work without permission, and credit.
+We also ask that you not share the ideas ideas presented in this class without credit.
+While the class website is public, we ask that you do not take things out of context.
+
+In addition, any tangible medium such as digital and physical documents are
 protected by copyright law as embodied in title 17 of the United
 States Code. These expressions include the work product of both:
 (1) your student colleagues (e.g., any assignments published here
 in the course environment or statements committed to text in a
 discussion forum); and, (2) your instructor (e.g., the syllabus,
-assignments, reading lists, and lectures). Within the constraints
-of &quot;fair use,&quot; you may copy these copyrighted
-expressions for your personal intellectual use in support of your
-education here in the UW. Such fair use by you does not include
-further distribution by any means of copying, performance or
-presentation beyond the circle of your close acquaintances,
-student colleagues in this class and your family. If you have any
-questions regarding whether a use to which you wish to put one of
-these expressions violates the creator's copyright interests,
-please feel free to ask the instructor for guidance.
-
-The essence of academic life revolves around respect not only for
-the ideas of others, but also their rights to those ideas. It is
-therefore essential that we take the utmost care that the ideas
-(and the expressions of those ideas) of others always be handled
-appropriately, and, where necessary, cited. When ideas or
-materials of others are used (particularly in your creative
-projects), they must be cited. The citation format is not that important -
-as long as the source material can be located and the citation
-verified, it's OK. In any situation, if you have a question,
-please feel free to ask.
-
-You must have the right to publish any of the images, videos,
-text, or other media on your creative sites. This means you may
-use:
-
-- Media you have created or generated yourself (i.e. pictures you have created or taken yourself, text you have written yourself.)
--  Images that are in the public domain (something from Wikipedia), or something with a creative commons license that allows for reuse without explicit permission of the owner.
-  - [Creative Commons Kiwi](https://creativecommons.org/about/videos/creative-commons-kiwi/) is a really informative video on Creative Commons licensing.
-  - Instructions on how to search for images that are fair use are [here](https://support.google.com/websearch/answer/29508?hl=en).
-  - You must cite any works that you use that you did not generate yourself (although technically you only need to cite things that are [CC Attribution](https://creativecommons.org/licenses/by/2.0/). A handy site for knowing how to add your citations is
-  [here](https://wiki.creativecommons.org/wiki/Best_practices_for_attribution).
+assignments, reading lists, and lectures). 
 
+Within the constraints of <q>fair use,</q> you may copy these
+copyrighted expressions for your personal intellectual use in support
+of your education here in the UW. Such fair use by you does not
+include further distribution by any means of copying, performance or
+presentation beyond the circle of your close acquaintances, student
+colleagues in this class and your family. If you have any questions
+regarding whether a use to which you wish to put one of these
+expressions violates the creator's copyright interests, please feel
+free to ask the instructor for guidance.
 
-# Collaboration Policies
-
->      As a teacher, it’s not my goal in life to read a class’ worth of
->      programs on a topic that all basically look the same. My goal is
->      to assess whether you understand the material that we’ve taught in
->      class enough, and you demonstrate that to me through the work you
->      do. I can not assess that if you are turning in someone else’s
->      work.
->      - Dr. Tom Butler, Lakeside H.S.
-
-Computer science education is odd in that we expect you to turn in
-work that you do completely independently when in the
-&quot;real world&quot; that’s not how it works at all. In the real
-world, co-workers collaborate, bounce ideas off each other, they
-look up parts of solutions on the internet. But in the &quot;real
-world&quot; the people doing the work have years of experience,
-they have proved themselves to their teachers, co-workers and
-bosses to where they are at that moment and most importantly, they
-know <strong>how to evaluate which of the solutions they are
-receiving is an appropriate one to solve the task at hand.</strong>
-
-As your instructor, I need to be able to evaluate
-<strong>your</strong> work. Thus, unless otherwise specified all
-work in this and other CS classes must be your own. We realize you
-may look at other sources online to learn how to achieve new things, but we expect
-you to synthesize this information and not copy it directly. You
-should never copy (plagiarize) homework or code from another person
-in this school (past or present) or that you find online directly
-and submit it as your own work.
-
-All programming assignments must be completed individually. However,
-you **may discuss homework assignments with other students**
-(i.e. provide advice on difficult concepts, *etc*) and are encouraged
-to discuss class material, such as section exercises, lecture
-material, readings, etc. Even better if this takes place on Ed where other students can benefit and we can guide you as to what is supportive and what crosses the line to too much sharing.  You may also discuss Creative Projects with
-other students, as long as the code you write is entirely your own,
-and discussion with students on Creative Projects should never involve
-details of how to code a solution. Specifically, you must abide by the
-following:
-
-- You **may not use code directly from any external sources** (including copying lecture/lab material directly into a programming assignments).
-- You may not post your homework solutions on a publicly accessible (non-password-protected) web server or Git repository, during the course or after it has been completed. Please see the course website for acceptable ways to show your work to others.
-- You may not look at or use prior solutions from any source.
-- You must document substantive discussions (i.e. when a classmate's advice helped you with a difficult concept) 
-- You may not provide a classmate with actual code or with step-by-step instructions
-
-In short: you should think of most assignments in this class as assessments and as such, complete them independently - unless otherwise told.
-
-Under our policy, a student who gives inappropriate help is
-equally guilty with one who receives it. Instead of providing such
-help to a classmate, point them to other class resources such as
-lecture examples, OH, or a TA. You must take reasonable steps
-to ensure that your work is not copied by others, such as making
-sure to log out or lock shared computers, not leaving printouts of
-your code in public places, and not emailing code to other
-students or posting it on the web or public forums.
-
-We enforce our policies by running detection software during the
-quarter over all programs, including ones from past quarters.
-Please contact us if you are unsure whether a particular behavior
-falls within our policy.
-
-## If you make use of a source, or advice from a classmate
-
-Some portions of our projects will have a creative aspect to them. On occasion students may wish to use portions of sample code that has been obtained on our course website or others. In order to avoid academic misconduct for a Creative portion of your projects you must:
-
-- Ensure that substantive original work is submitted that can be evaluated by the course staff.
-- Cite the ideas or materials of others that are used. The citation format is not that important - as long as the source material can be located and the citation verified (a url in a comment is generally fine), it's OK.
-- Clearly indicate (e.g. with comments) which portions of your code are completely original and which are used or modified from external sources, if any code is used that builds off of/is inspired by external sources (e.g. adaption of an example exercise, online tutorial you find). We will only grade your original work. Note that solely changing identifier names or rearranging other source material is not considered your original work - see the examples of appropriate use below for details.
-
-A good analogy to this is if you were writing a history paper: You can use quotes in your paper as long as you give attribution to the sources of the quote, but you can not write a history paper out of the quotes of others (particularly with no citations).
-
-Some examples of appropriate use:
-
-- A student closely follows a tutorial to understand a new concept in Android Development (e.g.  animations). The student cites the tutorial they used in the file header then substantially modifies the tutorial code to include what is specified for the creative portion of the assignment, documenting which portions of the code are their own so TAs know which portions to grade (and to determine whether the material cited as being learned from the tutorial is sufficiently adapted to be considered the student’s own work).
-- A student is having difficulty styling their website. They look for a solution and find one on a site such as Stack Overflow. The student uses the code they find in their solution, documents that small piece of code was not their own with a comment that includes where it was found. The TAs will not use that portion of the code in grading.
-
-Students with questions about any specific situation should ask the instructor for clarification.
+# Appropriateness
 
-# Privacy
+Recall that one of our course policies is to engender an
+inclusive environment. As such it is important that you are thoughtful about
+what you say or write. Please make sure that 
+images and text you are using are <q>school appropriate</q> and follow
+the guidelines of expected behavior. If you have any questions,
+please do not hesitate to ask the TA or your instructor. Inappropriate work
+submitted may be ineligible for credit on that assignment.
 
-To support an academic environment of rigorous discussion and open
-expression of personal thoughts and feelings, we, as members of
-the academic community, must be committed to the inviolate right
-of privacy of our student and instructor colleagues. As a result,
-we must forego sharing personally identifiable information about
-any member of our community including information about the ideas
-they express, their families, lifestyles and their political and
-social affiliations. If you have any questions regarding whether a
-disclosure you wish to make regarding anyone in this course or in
-the university community violates that person's privacy interests,
-please feel free to ask the instructor for guidance.
-
-Knowingly violating any of these principles of academic conduct,
-privacy or copyright may result in University disciplinary action
-under the Student Code of Conduct.

assets/css/my-remark.scss

@@ -144,16 +144,14 @@ table tbody td {
 table thead {
     background: $dark;
     color: $light;
-    border-bottom: 4px solid $dark;
     border-left: none;
     border-right: none;
 }
 
-table thead th {
+table thead tr {
   font-size: 0.7em;
   font-weight: bold;
   color: $light;
-  border-left: 2px solid $dark;
 }
 table thead th:first-child {
   border-left: none;

assets/css/style.scss

@@ -307,6 +307,7 @@ table.schedule tfoot td {
 .page-header {
   background-image: linear-gradient(-120deg, $gradient-light, $gradient-dark, $gradient-dark);
   padding-top: 1rem;
+  padding-bottom: 1rem;
 }
 
 .project-tagline {

docs/covid/index.md

@@ -13,7 +13,7 @@ will update this page as rules/regulations change.
 
 
 # Masking
-Masking is currently optional, but ``recommended indoors'' according to the [UW face covering policy](https://www.ehs.washington.edu/covid-19-prevention-and-response/face-covering-policy). 
+Masking is currently optional, but ``strongly recommended indoors'' according to the [UW face covering policy](https://www.ehs.washington.edu/covid-19-prevention-and-response/face-covering-policy). 
 
 In this class, people in the room include those who are high risk with respect to COVID and people who live with vulnerable family members. To help support the needs of these individuals, masks will be available in the classroom as you enter, and if you would like to mask, you are welcome to make use of them. 
 

index.md

@@ -62,6 +62,8 @@ Taking a class is a big commitment, and you will work hard in this class. So we
 
 The only requirement for this class is that you are comfortable programming and picking up new languages and tools that you have not been exposed to before. You will have some control over this, however, basic web skills are likely to be useful. The primary programming project in this class is one you design yourself. 
 
+In addition, please familiarize yourself with the course [academic conduct]({{site.baseurl}}/academic-conduct.html) policy. We will grade on the assumption that you have read and understood it. 
+
 # Course Structure
 
 Many of the goals in this class center around learning by doing. This

schedule.md

@@ -323,7 +323,7 @@ Discussion of Disability Dongle Paper
 
 {: .activity} 7:00-7:40 Skill Goal: Various Techniques to Make Visualizations Accessible
 : **In class Exercise**
-- Implement a visualization using [VoxLens](https://github.com/athersharif/voxlens) following the documentation and examples on the GitHub repository.
+<!-- - Implement a visualization using [VoxLens](https://github.com/athersharif/voxlens) following the documentation and examples on the GitHub repository. -->
 - Use VoxLens from to test different modes of making visualizations accessible (Summary, Sonification, Question-and-Answer) on different visualization types.
 - Identify the utility of each mode with each visualization type–when is one mode more suitable than the other and why.
 - Identify accessibility measures for demographics other than screen-reader users.

slides/3dprinting.html

@@ -23,6 +23,7 @@ class:
 
 ## This is an important reminder
 ## Make sure zoom is running and recording!!!
+## Make sure captioning is turned on
 
 ---
 layout:false

slides/accessible-data-viz.html

@@ -21,21 +21,31 @@ class:
 
 ## This is an important reminder
 ## Make sure zoom is running and recording!!!
+## Make sure captioning is turned on
+
+
+---
+# COVID-19 Visualizations
+
+![:img Visualizations of COVID-19 cases taken from different websites including Seattle Times and King County’s official website,80%, width](img/data-equity/covid-19-viz.png)
 
 ---
 [//]: # (Outline Slide)
-# Learning Goals for Today
 
-- What are the commonly-used techniques to make data visualizations accessible?
+# Some questions to consider about information access
+
+Who can access visualizations that other people have created?
 
-- What are the pros and cons of these techniques?
+Who can create accessible information?
 
-- What are some of the nuances in making data visualizations accessible?
+How do we support the full data science lifecycle?
 
 ---
-# COVID-19 Visualizations
+# Who can Access Visualizations?
+
+Due to the inaccessibility of online data visualizations, screen-reader users answered questions **62%** less accurately and spent **211%** more time than non-screen-reader users *(Sharif, 2021)*.
 
-![:img Visualizations of COVID-19 cases taken from different websites including Seattle Times and King County’s official website,100%, width](img/data-equity/covid-19-viz.png)
+![:img Two different graphs comprising bar chart of Accuracy of Extracted Information per Visualization Library and interaction plot of Interaction Times per Visualization Library per participant group.,80%, width](img/data-equity/sru-viz-results.png)
 
 ---
 # How Can We Make These Visualization Accessible?
@@ -47,13 +57,13 @@ class:
 ---
 # Commonly-Used Techniques
 
-- Summary ("Alt-Text")
+Summary ("Alt-Text")
 
-- Sonification (Audio Graphs)
+Sonification (Audio Graphs)
 
-- Tables
+Tables
 
-- Braille Prinouts/3-D Printing
+Braille Prinouts/3-D Printing
 
 ---
 # Summary ("Alt-Text")
@@ -67,15 +77,19 @@ Alt-text is an HTML attribute that can provide a summary of the visualization an
 
 ---
 # Sonification (Audio Graphs)
-
 Sonification (or audio graphs) enable listeners to interpret data trends by mapping data to a musical scale.
 
-[Example](https://www.youtube.com/watch?v=gssjuSnMR60)
+
+.left-column40[
 
 **Benefits**
 - Can help users to understand overall trends
 
 - Already integrated into commerical products (e.g., Apple)
+]
+.right-column60[
+![:youtube Video of a graph being sonified, gssjuSnMR60]
+]
 
 ---
 # Tables
@@ -98,34 +112,35 @@ Tactile graphics and 3-D printed graphs are raised surfaces that enable blind an
 - Enables users to interact with the visualization without using a screen reader
 
 ---
-# Equality vs. Equity
-
-- What does **equal** access mean?
-
-- What does **equitable** access mean?
-
-- Are these techniques equitable?
+# Holistic vs. Drilled-Down Exploration
 
----
-# State of The World
+Screen-reader users perform a holistic exploration first and then drill-down into the data *(Sharif, 2021)*.
 
-Due to the inaccessibility of online data visualizations, screen-reader users answered questions **62%** less accurately and spent **211%** more time than non-screen-reader users *(Sharif, 2021)*.
+.left-column50[
+- Summary ("Alt-Text")
+- Sonification (Audio Graphs)
+- Tables
+- Braille Prinouts/3-D Printing
+]
+.right-column50[
 
-![:img Two different graphs comprising bar chart of Accuracy of Extracted Information per Visualization Library and interaction plot of Interaction Times per Visualization Library per participant group.,100%, width](img/data-equity/sru-viz-results.png)
+- Which of these techniques enable **holistic** exploration?
+- Which of these techniques enable **drilled-down** exploration?
+]
 
 ---
-# Holistic vs. Drilled-Down Exploration
+# Supporting Interaction
 
-Screen-reader users perform a holistic exploration first and then drill-down into the data *(Sharif, 2021)*.
-
-- Which of these techniques enable **holistic** exploration?
+Highly relevant even when just *using* a visulazation
+- When interacting with static visualizations, sighted people intuitively construct complex visual queries [Ware, 2019]. 
+- Interactive charts afford selection, filtering & more [Yi, IEEE TOVCG, ’07]
 
-- Which of these techniques enable **drilled-down** exploration?
+Supporting this requires going past verbal descriptions and tactile graphics
 
 ---
-# VoxLens 
+# [VoxLens](https://github.com/athersharif/voxlens).
 
-VoxLens is an open-source JavaScript plug-in that improves the accessibility of online data visualizations for screen-reader users using a multi-modal approach, requiring only a single line of code for integration into online data visualizations. 
+[VoxLens](https://github.com/athersharif/voxlens) is an open-source JavaScript plug-in that improves the accessibility of online data visualizations for screen-reader users using a multi-modal approach, requiring only a single line of code for integration into online data visualizations. 
 
 VoxLens offers three modes: 
 
@@ -135,6 +150,11 @@ VoxLens offers three modes:
 
 .center[https://github.com/athersharif/voxlens]
 
+---
+# VoxLens
+
+![:img Quering Voxlens. First query is "What's the maximum data point in the graph?" with answer "I understood you're looking for maximum. Maximum value of Total Crime for Crime Neighborhood is 48900 belonging to Downtown Commercial." Second query is "What about average and variance?" with answer "I heard you asking about the average and variance. Average of Total Crime for Crime Neighborhood is 8870. Variance of Total Crime for Crime Neighborhood is 78800000" ,50%, width](img/data-equity/voxlens-interactive.png)
+
 ---
 # Evaluation of VoxLens
 
@@ -143,27 +163,149 @@ VoxLens improved the accuracy of screen-reader users by **122%** and interaction
 ![:img Two different graphs comprising bar chart of Accuracy of Extracted Information per Visualization Library and interaction plot of Interaction Times per Visualization Library per participant group.,100%, width](img/data-equity/voxlens-viz-results.png)
 
 ---
-# Class Activity
+[//]: # (Outline Slide)
 
-**Activitiy**
-- Break into small groups.
-- Implement a visualization using [VoxLens](https://github.com/athersharif/voxlens).
-- Use VoxLens from to test different modes of making visualizations accessible (Summary, Sonification, Question-and-Answer).
+# Some questions to consider about information access
 
-**Goals**
-- Identify the utility of each mode with each visualization type–when is one mode more suitable than the other and why.
-- Identify accessibility measures for users other than screen-reader users.
-- Create a presentation (2-3 slides) to share findings
+Who can access visualizations that other people have created?
 
-.center[https://github.com/athersharif/voxlens]
+**Who can create accessible information?**
+
+How do we support the full data science lifecycle?
+
+---
+# *Who* Can Creating Accessible Visualizations
+
+Can BLV people create new information 
+- Are they included in the employment opportunities and learnings associated with data science?
+- Can they access information when someone didn’t already decide it needs to be accessible
+  - Maps and other individualized needs
+  - Live data exploration
+
+---
+# Non Programmers 
+
+.left-column40[
+## Maptimizer
+ 
+![:img A 3d printed tactile map of the UW campus showing the fountain and the grassy area around it,100%, width](img/data-equity/maptimizer.jpg)
+(Hofmann, CHI 2022) 
+]
+
+.right-column60[
+## Generative Design 
+*Generative design* is a common approach to support end users (used a lot for fabrication)
+
+Lets each user specify their information priorities and preferred representations in familiar terms through a simple GUI
+
+]
+
+---
+# What About Direct Authoring?
+
+We should also support more advanced creation
+- In Kim et al.’s survey of 56 accessible visualization papers published between 1999 and 2020 only two papers supported interactive authoring of charts by BVI users [Kim, CGF ‘21]
+
+Equivalent experiences is an important access goal
+- Same information conveyed
+- Same *interactive experience*
+
+---
+# PSST
+
+Based on RxJS: everything is a stream
+
+.left-column40[
+<div class="mermaid">
+
+graph TD
+    LS[Sensor] --> DS[DataSink]
+    OE[OutputEngine] --> DS
+    DS -->|Sensor values| NH[Handler]
+    NH -->|converts sensor values to appropriate format| O[Output]
+	
+</div>
+]
+
+---
+# PSST Example
+
+Sonifying a light sensor
+.left-column40[
+<div class="mermaid">
+
+graph TD
+    LS[Light Sensor] --> DS[DataSink]
+    OE[OutputEngine] --> DS
+    DS -->|Light sensor values| NH[NoteHandler]
+    NH -->|converted light sensor values to audible range| O[NoteOutput]
+	
+</div>
+]
+.right-column60[
+![:youtube A sound can be heard with the pitch rising and lowering as a light moves closer and further from the circuit board, aSe-QkSl12c]
+]
+
+<!-- --- -->
+<!-- # PSST  -->
+
+<!-- PSST allows BVI developers to make their own data visualizations -->
+
+<!-- <div class="mermaid"> -->
+<!-- sequenceDiagram -->
+<!--     Lightsensor->>+DataSink: New Value as Datum -->
+<!--     OutputEngine->>+DataSink: Status is "Playing" -->
+<!--     DataSink-\->>+NoteHandler: Status, Datum -->
+<!--     NoteHandler->>+NoteOutput: Status, Datum adjusted using Mel Scale -->
+<!--     NoteOutput->>+User: Plays note if status is "Playing" -->
+<!-- </div> -->
+
+---
+# Code or GUI Configuration
+
+.column[
+GUI automatically created from spec
+
+User can select, parameterize and attach outputs to streaming data
+]
+.column[
+![:youtube A user configuring a sonification using the accessible PSST GUI while a screenreader plays a description, xl70zqfW2H0]
+
+]
+
+--
+.column[
+.quote[I’ve seen these sonifications for decades . . . but I’ve never seen anything nice in a dashboard like this that lets you add and remove multiple ones and tweak the parameters of each.]
+]
 
 ---
 [//]: # (Outline Slide)
-# Learning Goals for Today
 
-- What are the commonly-used techniques to make data visualizations accessible?
+# Some questions to consider about information access
+
+Who can access visualizations that other people have created?
+
+Who can create accessible information?
+
+**How do we support the full data science lifecycle?**
+- Data collection
+- Data cleaning
+- Modeling and Analysis
+- Representation
 
-- What are the pros and cons of these techniques?
+---
+# Data Science Accessibility
+
+Very much still a wide open area, at least for BLV people
+
+- Data collection -- not particularly supported (even coding tools very inaccessible)
+- Data cleaning -- not studied to my knowledge
+- Modeling and Analysis -- Common tools like Jupyter Notebooks wholly inaccessible. Venkatesh (first author of PSST) is starting to work on this
+- Representation -- Accessible visualization & Data physicalization are the big opportunities
+
+---
+# One last PSST Video
+... and then on to our guest!
 
-- What are some of the nuances in making data visualizations accessible?
+![:youtube A user turns a handle on a music box. A laser cut strip containing data moves through producing notes, EKaaP0TKQog]
 

slides/bias-in-machine-learning.html

@@ -21,6 +21,7 @@ class:
 
 ## This is an important reminder
 ## Make sure zoom is running and recording!!!
+## Make sure captioning is turned on
 
 ---
 [//]: # (Outline Slide)
@@ -54,10 +55,94 @@ But *how*?
 
 - How does Machine Learning solve a problem?
 
+
 ---
-# Components of Machine Learning
+# Helping Computers Learn Patterns
 
-- Collect data (and lots and lots of it!)
+.left-column50[
+![:fa bed, fa-7x]
+]
+.right-column50[
+## How might you recognize sleep?
+
+- Can you come up with a yes/no question or a set of categories or simple description of sleep?
+   - Sleep quality?
+   - Sleep start/end? 
+- What data would you learn from?
+- How might you need to take disabilites into account?
+]
+???
+
+(sleep quality? length?...)
+
+How to interpret sensors?
+
+---
+# How do we program this?
+
+Old Approach: Create software by hand
+- Use libraries (like JQuery) and frameworks
+- Create content, do layout, code up functionality
+- Deterministic (code does what you tell it to)
+
+New Approach: Collect data and train algorithms
+- Will still do the above, but will also have some functionality based
+on ML
+- *Collect lots of examples and train a ML algorithm*
+- *Statistical way of thinking*
+
+---
+# Shift in Approaches
+
+.left-column50[
+## Old style of app design
+<div class="mermaid">
+graph TD
+
+I(Input) --Explicit Interaction--> A(Application)
+A --> Act(Action)
+
+classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
+
+class U,C,A,I,S,E,Act,Act2 normal
+</div>
+]
+
+
+--
+count: false
+.right-column50[
+## New style of app design
+<div class="mermaid">
+graph TD
+
+U(User) --Implicit Sensing--> C(Application)
+S(System) --Implicit Sensing--> C
+E(Environment) --Implicit Sensing--> C
+C --> Act2(Action)
+
+classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
+
+class U,C,A,I,S,E,Act,Act2 normal
+</div>
+]
+---
+# This is *Machine Intelligence*
+
+Often used to process sensor data
+
+Goal is to develop systems that can improve
+performance with more experience
+- Can use "example data" as "experience"
+- Uses these examples to discern patterns
+- And to make predictions
+
+Not really intelligent, just my word for Machine Learning, AI, Neural Programming, etc etc
+
+---
+# Components of Machine Intelligence
+
+- **Collect data (and lots and lots of it!)**
 
 - Discern patterns
 
@@ -73,49 +158,370 @@ But *how*?
 - Who do we collect data from?
 
 ---
-# "Goodness" of Data
+# Problems with Data
+- System timeouts that are trained on movement speeds of <q>typical</q> people
+- Biometrics that cannot function on a person who isn't still for long enough
+- Inferencing about people that doesn't account for height; stamina; range of motion; or AT use (e.g. wheelchairs)
+
+When groups are historically
+marginalized and underrepresented, this is
+.quote[imprinted in the data that shapes AI
+systems... Those who have borne discrimination in the past are most at risk of harm from
+biased and exclusionary AI in the present. (Whittaker, 2019)] 
+
+--
+This can cascade -- e.g. measurement bias can exacerbate bias downstream. For example, facial mobility, emotion expression, and facial structure impact detection and identification of people; body motion and shape impact activity detection; etc. 
+
+---
+# How might we address bias/fairness in data sets
+
+We need to know it is there (Aggregate metrics can hide  performance problems in under-represented groups)
+
+We need to be careful not to eliminate, or reduce the influence, of outliers if that erases disabled people from the data because of the heterogeneity of disability data.
+
+---
+# Approaches to measuring fairness 
+
+We may need to rethink <q>fairness</q> in terms of individual rather than group outcomes, and define metrics that capture a range of concerns
+- Movement speed might favor a wheelchair user
+- Exercise variety might favor people who do not have chronic illness
+- Exertion time might covers a wide variety of different types of people.
+
+Defining such unbiased metrics requires careful thought and domain knowledge, and scientific research will be essential to defining appropriate procedures for this.
+
+---
+# Small Group Discussion [Post on Ed]({{site.discussion}}2514887)
+
+Who might be excluded in the data set you found?
+
+How was fairness measured in the data set you found, if it was discussed?
+
+How would you go about testing for fairness in that data?
+
+
+---
+# Best Practices For Data Fairness
+
+How do we motivate and ethically compensate disabled people to give their data?
+
+What should we communicate at data collection time? 
+
+Is the data collection infrastructure accessible? Does it protect sensitive information about participants adequately given the heterogeneous nature of disability? 
+
+
+Does the meta data collected oversimplify disability? Who is labeling the data and do the have biases affecting labeling?
+  - Whittaker (2019) discusses the example of clickworkers who label people
+as disabled <q>based on a hunch</q>. 
+
+---
+# Components of Machine Intelligence
+
+- Collect data (and lots and lots of it!)
+
+- **Discern patterns**
+
+- Make predictions
+
+
+---
+# Two main approaches
+
+![:fa eye] *Supervised learning* (we have lots of examples of what should be
+ predicted)
+
+![:fa eye-slash] *Unsupervised learning* (e.g. clustering into groups and inferring what
+they are about)
+
+![:fa low-vision] Can combine these (semi-supervised)
+
+![:fa history]  Can learn over time or train up front
+
+---
+# Our Focus: Supervised Learning
+
+![:fa eye] *Supervised learning* (we have lots of examples of what should be
+ predicted)
+
+![:fa eye-slash] *Unsupervised learning* (e.g. clustering into groups and inferring what
+they are about)
+
+![:fa low-vision] Can combine these (semi-supervised)
+
+![:fa history]  Can learn over time or train up front
+
+---
+# Supervised Learning
+
+.left-column50[
+## Training Process
+
+<div class="mermaid">
+graph TD
+
+L(Label) --> MI(Training Algorithm)
+D(Input Data) -- Extract Features--> MI
+MI --> C(Symbolic Predictor)
+
+classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
+
+class D,U,C,A,I,S,E,Act,Act2 normal
+</div>
+]
+
+
+.right-column50[
+## Extracting Features
+
+Symbolic requires feature engineering (humans deciding how to *summarize* data using features. Tends to be more *interpretable* (you can figure out why they make predictions)
+
+]
+
+---
+# Supervised Learning
+
+.left-column50[
+## Training Process
+
+<div class="mermaid">
+graph TD
+
+L(Label) --> MI(Training Algorithm)
+D(Input Data) --> MI
+MI --> C(Neural Predictor)
+
+classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
+
+class D,U,C,A,I,S,E,Act,Act2 normal
+</div>
+]
+
+
+.right-column50[
+## Designing Networks
+
+Neural approaches (e.g. ChatGPT) use massive amounts of data to train a network according to base principales. Designing the right network is critical. Cannot be sure *why* they make the predictions they do. 
+]
+
+---
+# Supervised Learning
+
+.left-column50[
+## Training Process
+
+<div class="mermaid">
+graph TD
+
+L(Label) --> MI(Training Algorithm)
+D(Input Data) --> MI
+MI --> C(Symbolic/Neural Predictor)
+
+classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
 
-Let's consider this example: **We wish to train a ML model to recognize if a given picture is that of a nurse.**
+class D,U,C,A,I,S,E,Act,Act2 normal
+</div>
+]
+
+
+.right-column50[
+## Prediction Process
 
-What steps do we take?
+<div class="mermaid">
+graph TD
 
-Is the data "good"?
+D(Input Data) --> C(Symbolic/Neural Predictor)
+C --> P(Prediction)
+
+classDef normal fill:#e6f3ff,stroke:#333,stroke-width:2px;
+
+class P,D,U,C,A,I,S,E,Act,Act2 normal
+</div>
+]
 
-What other examples can we think of?
 
 ---
-# "Goodness" of Data
+# How do we Evaluate Predictors?
 
+Compare to Prior probabilities
+- Probability before any observations (ie just guessing)
+- Ex. ML classifier to guess if an animal is a cat or a ferret based on the ear location
+ - Assume all pointy eared fuzzy creatures are cats (some percentage will be right)
+
+Compare to simplistic algorithms
+- Ex. Classifying cats vs ferrets based on size
+- Your model needs to do better than these too
+
+Surprising how often this doesn't happen in published work/before deployment
+
+???
+We did this to study gender's impact on academic authorship; doctors reviews
+
+---
+# Adding Nuance
+.left-column50[
+## <q>Confusion Matrix</q>
 ![:img Confusion matrix of a machine learning model,100%, width](img/data-equity/ml-faulty.png)
+]
+.right-column50[
+Don't just measure accuracy (percent correct)
+
+Lots of other metrics based on false positives and negatives
+- Precision = TP / (TP+FP) Intuition: Of the positive items, how many right?
+- Recall = TP / (TP+FN) Intuition: Of all things that should have been positive, how many actually labeled correctly?
+- ... Many More
+]
 
 ---
-# Disability Bias in Machine Learning 
+# Using Proper Methods
 
-Data sets used to train biometric systems are biased: they rarely if ever have a comprehensive
-representation of the range of people they might encounter. *(Mankoff, 2021)*
+.left-column50[
+**Symbolic Methods Can Easily Overfit**
 
-- Human bias 
+When your ML model is too specific for data you have, it might not generalize well
 
-- Machine bias
+Best test is a data set you haven't seen before
 
-- Underrepresentation
+![:img overfitting is illustrated as a line snacking between data points to minimize error instead of smoothly rising among them , 80%,width](img/data-equity/overfitting.png)
 
-- Measurement Errors
+]
+
+.right-column50[
+**Neural Methods Can Have Hidden Biases** 
+
+![:img A headline from the Verge stating that Twitter taught Microsoft's AI Chatbot to be a racist asshole in less than a day, 80%,width](img/data-equity/racist-chatbot.png)
+
+]
 
 ---
-# Minimizing Bias
+# Disability Biases to Watch Out For
 
-- Appropriate representation
+Norms are baked deeply into algorithms which are designed to learn about the most common cases. As human judgment is increasingly replaced by biometrics, *norms* become more strictly enforced. 
+- Do outliers face higher error rates? 
+- Do they disproportionately represent and misrepresent people with disability?
+- How does this impact allocation of resources?
 
-- Large training data 
+---
+# How does norming harm people with disabilities?
 
-- Validating the models 
+Machine intelligence  already being used to track  allocation of assistive technologies, from CPAP machines for people with sleep apnea (Araujo 2018) to prosthetic legs (as described by Jullian Wiese in
+Granta and uncovered in Whittaker et al 2019), deciding who is <q>compliant enough</q> to deserve them. 
 
-- Montoring the performance of the models
+Technology may also fail to recognize that a disabled person is even present (Kane, 2020),  thus <q>demarcating what it means to be a legible human and
+whose bodies, actions, and lives fall outside... [and] remapping and calcifying the boundaries
+of inclusion and marginalization</q> (Whittaker, 2019). 
 
 ---
-# Discussion
+# How does norming harm people with disabilities?
 
-- Identify biases in existing ML-based tools/platforms
+Many biometric systems gatekeep access based on either individual identity, identity as a human, or class of human, such as <q>old enough to buy cigarettes.</q>
+Examples:
+- a participant having to falsify data because <q>some apps [don’t allow] my height/weight combo for my age.</q> (Kane (2020))
+- a person who must ask a stranger to ‘forge’ a signature at the grocery store <q>.. because I can’t reach [the tablet]</q> (Kane (2020))
+- at work, activity tracking may define <q>success</q> in terms that exclude disabled workers. (may also increase the likelihood of work-related disability, by forcing workers to work at maximal efficiency)
 
-- Identify ways to make ML models more inclusive in your area of expertise
+---
+# Components of Machine Intelligence
+
+- Collect data (and lots and lots of it!)
+
+- Discern patterns
+
+- **Make predictions**
+
+---
+# Concerns at Prediction Time
+
+Denial of insurance and medical care, or threaten employment (Whittaker,
+2019, p. 21).
+- HireVue, an AI based video
+interviewing company has a patent on file to detect disability (Larsen, 2018). 
+- This is illegal under the ADA, which
+   - forbids asking about disability
+status in a hiring process (42 U.S.C. § 12112(a)) 
+   - forbids <q>using qualification
+standards, employment tests or other selection criteria that screen out or tend to screen out
+an individual with a disability</q> (42 U.S.C. § 12112(b)(6)). 
+
+---
+# Concerns at Prediction Time
+
+Denial of insurance and medical care, or threaten employment
+
+Disability identification 
+- Examples: detect Parkinsons from gait (Das, 2012), and mouse movement (Youngmann,
+2019); detecting autism from home videos (Leblanc, 2020). 
+- What are the ethics of doing this without consent?
+- Many of these algorithms encode medical model biases
+
+Relatedly, failure to identify disability 
+- Legally under the ADA, if you are treated as disabled, you are disabled. Yet biometrics cannot detect how people are treated. 
+
+---
+# Concerns at Prediction Time
+
+Denial of insurance and medical care, or threaten employment 
+
+Disability Identification / Failure to Identify
+
+Apps that Harm
+- Example: Training behaviors in <q>support</q> of autistic individuals without regard to debates about agency and independence of the target audience [Demo, 2017]; 
+- As with regular accessibility apps, AI based aps can harm, be disability dongles, etc
+- As with regular apps, AI based apps may not be accessible
+
+---
+
+# Concerns at Prediction Time
+
+![:img Three news headlines-- On Orbitz Mac Users Steered to Pricier Hotels; Google's algorithm shows prestigious job ads to men but not to women; Racial bias alleged in Google's add results, 60%,width](img/data-equity/bias.png)
+
+---
+# Concerns at Prediction Time
+Denial of insurance and medical care, or threaten employment 
+
+Disability Identification / Failure to Identify
+
+Apps that Harm
+
+AI with Baked in Biases
+- Consequences of biased data and lack of control over training results more nuanced than just accuracy (as with headlines we just read)
+- Privacy can also be a concern. 
+  - For rare conditions, an algorithm may learn
+to recognize the disability, rather than the individual, reducing
+security when used for access control, allowing multiple people with
+similar impairments to access the same data.
+
+---
+# Concerns at Prediction Time
+
+Denial of insurance and medical care, or threaten employment 
+
+Disability Identification / Failure to Identify
+
+Apps that Harm
+
+AI with Baked in Biases
+
+Transparency and Accountability
+- Power differences between builders and users 
+- Representation of disabled people among builders
+- Algorithms that are not *interpretable* or *correctable*
+- Users of algorithms whose use them to enforce larger societal harms
+
+---
+# Small Group Discussion [Post on Ed]({{site.discussion}}2515387)
+
+Revisit the data set you chose
+
+Do you know what sort of predictions it was used for if any?
+
+What possible harms could be done with those predictions?
+
+Reminder of our list
+- Denial of insurance and medical care, or threaten employment 
+- Disability Identification / Failure to Identify
+- Apps that Harm
+- AI with Baked in Biases
+- Transparency and Accountability
+
+
+---
+# End of Deck