6 merge requests!12Accessibility website jen summer work,!9Q access 24 sp,!8Q access 24 sp,!5Latest update from Spring 2023,!4Latest update after pmp class,!3Updated content for week 3 and week 4 (week 4 still needs work)
Objective:Teach machines to recognise non-speech sounds that occur around you and visualize these recognised sounds.
---
## Timeline (tentative)
-<strong>In-class check-in</strong> on May 3, 2021.
-<strong> Final deliverable</strong> in class on May 5, 2021
# Learning goals
In this homework, you will do two things:
1. Use a Jupyter notebook (provided to you on canvas) that contains code to train a machine learning model to recognise sounds and build visualizations to display the recognised sounds. We will post the notebook on canvas, and you are strongly encouraged to host the notebook on Google Collab. Login to your UW CSE provided google account, upload the notebook on your drive, and open the notebook in google colab on your browser. [Here is a getting started tutorial on colab](https://colab.research.google.com/notebooks/intro.ipynb#).
2.Caption a video, read related papers, and reflect on your experience captioning these videos. You should submit a caption file in a standard format like wsb or srt. Please review some helpful links in the deliverables section of this page to create a caption file in these standard formats. You are allowed to start with an AI-based service to generate captions, however, you should actively fix erroneous captions if you are using the AI-based service.
1. Design a visualization and provide a rationale for its purpose and value in code/sketch
2. Understand the difference between text for captioning speech and representations of nonspeech audio through the act of captioning a video that includes both speech and nonspeech audio
2.Exploring how context may impact representation through the readings
## Deliverables
You will submit the following
* A completed notebook or a link to the completed notebook with sufficient permissions to the staff to access, run and evaluate it.
* Link to, or the original file of the video you chose to caption.
* A link to, or the original caption file in a standard format like SBV or SRT. [this blogpost from rev.com](https://www.rev.com/blog/close-caption-file-format-guide-for-youtube-vimeo-netflix-and-more) is a good summary of different formats. Once you have captions, making a caption file in one of these standard formats should not be hard. For example, here is a [tutorial on using youtube to add captions to your videos](https://support.google.com/youtube/answer/2734796?hl=en).
# Activities
In this homework, you will do three things:
1. Build a visualization of non-speech audio.
2. Caption a video.
3. Read related papers.
4. Answer Reflection Questions
All of these are described in more depth in the Jupyter Notebook
## Build a visualization of non-speech audio
We will provide a Jupyter notebook (linked on canvas) via Google Colaboratory that is pre-loaded with code to train a machine learning model to recognize sounds. Everything you need is in the notebook.
To access it: Open it in your browser. You will be prompted to copy it. Then login to your UW CSE provided google account and open the notebook in google colab on your browser. [Here is a getting started tutorial on colab](https://colab.research.google.com/notebooks/intro.ipynb#).
## Caption a video
You should submit a caption file in a standard format like wsb or srt. You are allowed to start with an AI-based service to generate captions, however, you should actively fix erroneous captions if you are using the AI-based service.
[This blogpost from rev.com](https://www.rev.com/blog/close-caption-file-format-guide-for-youtube-vimeo-netflix-and-more) is a good summary of different formats. Once you have captions, making a caption file in one of these standard formats should not be hard. For example, here is a [tutorial on using youtube to add captions to your videos](https://support.google.com/youtube/answer/2734796?hl=en).
Note: You may not be able to upload captions to a youtube video you don't own, but you can still upload the caption file to Canvas to turn it in. Please use SBV or SRT format.
## Remainder of homework
All of the papers to read and the reflection questions can be found in the Jupyter notebook.
We have allowed maximum flexibility to upload these deliverables to canvas. Please upload multiple files, or use the comments section to submit links. Please indicate how exactly have you submitted the deliverables using the comment functionality of canvas once you submit your work.