diff --git a/assignments/hw2-soundRecognition.md b/assignments/hw2-soundRecognition.md index d3700a58bf4166a3b9d6385451074aca4062cde9..86be139b8d4e2dd6fa1eebe08c7bb575e7cc6346 100644 --- a/assignments/hw2-soundRecognition.md +++ b/assignments/hw2-soundRecognition.md @@ -9,24 +9,41 @@ assigned: April 26, 2021 due: May 5, 2021 -revised: April 25, 2021 +revised: April 26, 2021 Objective: Teach machines to recognise non-speech sounds that occur around you and visualize these recognised sounds. --- -## Timeline (tentative) -- <strong>In-class check-in</strong> on May 3, 2021. - - <strong> Final deliverable</strong> in class on May 5, 2021 +# Learning goals -In this homework, you will do two things: -1. Use a Jupyter notebook (provided to you on canvas) that contains code to train a machine learning model to recognise sounds and build visualizations to display the recognised sounds. We will post the notebook on canvas, and you are strongly encouraged to host the notebook on Google Collab. Login to your UW CSE provided google account, upload the notebook on your drive, and open the notebook in google colab on your browser. [Here is a getting started tutorial on colab](https://colab.research.google.com/notebooks/intro.ipynb#). -2. Caption a video, read related papers, and reflect on your experience captioning these videos. You should submit a caption file in a standard format like wsb or srt. Please review some helpful links in the deliverables section of this page to create a caption file in these standard formats. You are allowed to start with an AI-based service to generate captions, however, you should actively fix erroneous captions if you are using the AI-based service. +1. Design a visualization and provide a rationale for its purpose and value in code/sketch +2. Understand the difference between text for captioning speech and representations of nonspeech audio through the act of captioning a video that includes both speech and nonspeech audio +2. Exploring how context may impact representation through the readings -## Deliverables -You will submit the following -* A completed notebook or a link to the completed notebook with sufficient permissions to the staff to access, run and evaluate it. -* Link to, or the original file of the video you chose to caption. -* A link to, or the original caption file in a standard format like SBV or SRT. [this blogpost from rev.com](https://www.rev.com/blog/close-caption-file-format-guide-for-youtube-vimeo-netflix-and-more) is a good summary of different formats. Once you have captions, making a caption file in one of these standard formats should not be hard. For example, here is a [tutorial on using youtube to add captions to your videos](https://support.google.com/youtube/answer/2734796?hl=en). +# Activities + +In this homework, you will do three things: +1. Build a visualization of non-speech audio. +2. Caption a video. +3. Read related papers. +4. Answer Reflection Questions + +All of these are described in more depth in the Jupyter Notebook + +## Build a visualization of non-speech audio + +We will provide a Jupyter notebook (linked on canvas) via Google Colaboratory that is pre-loaded with code to train a machine learning model to recognize sounds. Everything you need is in the notebook. + +To access it: Open it in your browser. You will be prompted to copy it. Then login to your UW CSE provided google account and open the notebook in google colab on your browser. [Here is a getting started tutorial on colab](https://colab.research.google.com/notebooks/intro.ipynb#). + +## Caption a video +You should submit a caption file in a standard format like wsb or srt. You are allowed to start with an AI-based service to generate captions, however, you should actively fix erroneous captions if you are using the AI-based service. + +[This blogpost from rev.com](https://www.rev.com/blog/close-caption-file-format-guide-for-youtube-vimeo-netflix-and-more) is a good summary of different formats. Once you have captions, making a caption file in one of these standard formats should not be hard. For example, here is a [tutorial on using youtube to add captions to your videos](https://support.google.com/youtube/answer/2734796?hl=en). + +Note: You may not be able to upload captions to a youtube video you don't own, but you can still upload the caption file to Canvas to turn it in. Please use SBV or SRT format. + +## Remainder of homework +All of the papers to read and the reflection questions can be found in the Jupyter notebook. -We have allowed maximum flexibility to upload these deliverables to canvas. Please upload multiple files, or use the comments section to submit links. Please indicate how exactly have you submitted the deliverables using the comment functionality of canvas once you submit your work. \ No newline at end of file diff --git a/assignments/index.md b/assignments/index.md index 7ccc771b9207959625c74e605d3ae44942c6b85e..3ccea3d0d7af06235f6aac2cf2b6a1676421bce3 100644 --- a/assignments/index.md +++ b/assignments/index.md @@ -22,7 +22,7 @@ questions or run into issue, please contact the course staff. | [HW1: Improve a screen reader]({{site.baseurl}}/assignments/hw1-nvda.html) | | | | In Class Activity: Using switch access on your phone | | | | [Activity3: Describe pictures and videos]({{site.baseurl}}/assignments/act3-describing-video-image.html) | | | -| [HW2: teach machines to recognize sounds and visualize them]({{site.baseurl}}/assignments/hw2-soundRecognition.html) -- spec coming soon, timeline updated. | | | +| [HW2: Speech and Nonspeech Audio Visualization]({{site.baseurl}}/assignments/hw2-soundRecognition.html) | | | | [Final project: Make the World more accessible]({{site.baseurl}}/assignments/project.html) | | |