Automatic Zoom Transcriptions
As classes transition online for the fall, universities are working to lessen the impact on students’ learning and the strain on instructors working to adapt to these new circumstances. Accessibility is one of the major areas disrupted by this change, and the UBC CIC investigated this challenge with the tools and solutions available to us.
Overview
At the request of the University of British Columbia‘s Provost’s office, the CIC team explored a solution that will assist students in synchronous online lectures by providing a transcription of the lecture in real time. This solution will benefit anyone who is unable to participate synchronously, as well as students who benefit from seeing course material in written form.
Approach
The solution provides a consistent experience for students when instructors deliver course materials through Zoom. Instructors can use this tool to employ live Closed Caption transcriptions. The transcriptions are displayed in real time in the Zoom interface. Faculty and students also have the ability to download the time-stamped transcriptions upon completion of the lecture.
The solution utilizes Amazon Transcribe, AWS Amplify, AWS API Gateway and AWS Lambda, and it is cost-effective to deploy at rates of $1.44/hour for Amazon Transcribe. Amplify build and Cloud Formation are used in combination to deploy this solution.
In order to enable the closed captioning, users will follow these steps:
- Sign in to the Zoom web portal.
- In the navigation panel, click Settings.
- Click the Meeting tab.
- Verify that Closed Caption is enabled — if not, click the toggle to enable it.
- If a verification dialog displays, click Turn On to verify the change.
- Enable Save Captions to allow participants to save transcripts at the end of the Zoom session.
The speaker’s microphone captures audio and forwards it to the Amazon Transcribe service via websocket. This does not capture all speakers in the Zoom lecture, but it is possible to inadvertently do so if the instructor has their speakers on — the browser microphone could capture other audio and relay it to the AWS transcribe service. If an instructor does wish to capture all voices in a session, they would need to consider live-streaming the session and processing the captured audio and video.
Once the IT department has implemented the tool, instructors can go to the tools URL and paste their Zoom CC API token. Allowing microphone access will begin to forward the audio to the AWS Transcribe service for transcription. Once audio begins to transcribe, the Closed Captions will be overlaid on the video. Clicking on the ‘Closed Caption’ icon in the meeting task bar will give the option to show a full transcript in the sidebar. If save captions have been enabled, users will be prompted to save the transcript at the end of the Zoom meeting.
By saving the transcripts and uploading them to the course LMS (learning management system), instructors have the option of making the lecture transcriptions available to students. If the “Save Transcript” option is enabled, instructors are also able to add the transcript to their institution LMS for reference. UBC Zoom Transcription
The cost breakdown works out to less than $1.50/hr to run:
Amazon Transcribe. $0.0004 per second. Usage is billed in one-second increments.
https://aws.amazon.com/transcribe/pricing/
AWS API Gateway:
REST API – First 333 million requests $3.50
HTTP API – First 300 million requests $1.11
https://aws.amazon.com/api-gateway/pricing/
AWS Lambda:
Requests $0.20 per 1M requests
Github repo: https://github.com/UBC-CIC/amazon-transcribe-captions-for-zoom
The project is largely based on code from the AWS Sample – Amazon Transcribe Websocket: https://github.com/aws-samples/amazon-transcribe-websocket-static
FAQs
Q: How do I deploy this?
A: A combination of an Amplify build and Cloud Formation is used to deploy this solution.
Q: What settings need to be enabled in Zoom for this to work
A: 1. The Account admin has enabled the ability for users to enable closed captions.
2. Users to enable the closed captioning for your own use:
– Sign in to the Zoom web portal.
– In the navigation panel, click Settings.
– Click the Meeting tab.
– Verify that Closed Caption is enabled.
– If the setting is disabled, click the toggle to enable it.
– If a verification dialog displays, click Turn On to verify the change.
– Enable Save Captions to allow participants to save transcripts at the end of Zoom session
Q: Does this capture all speakers in the Zoom lecture?
A: No. This is using the speaker’s microphone to capture audio and forward to the Amazon Transcribe service via websocket it is possible inadvertently if the speaker has their speakers on for the browser microphone to capture other audio and relay to the AWS transcribe service.
If you did want to capture all voices in a session you would need to consider live streaming the session and processing the captured audio and video. Please see this reference solution if this is something that interests you: https://aws.amazon.com/solutions/implementations/live-streaming-with-automated-multi-language-subtitling/
Q: How do faculty use this for their lectures?
A: Once the IT department has implemented the tool, go to the tools URL and paste your Zoom CC API token. Allowing microphone access will start forwarding the audio to the AWS Transcribe service for transcription.
Q: How do you get the CC API Token from Zoom to use in the Real Time Transcription web app?
A: After launching a new Zoom meeting on the meeting room task bar click on the ‘Closed Caption’ icon and ‘Copy the API token’ this will be used in the AWS Real Time Transcribe app.
Q: What browser settings need to be enabled to record?
A: When using the application users will have to allow the browser to access their microphone.
Q: How do users show the CC Transcriptions in Zoom?
A: Once audio begins to transcribe the Closed Captions will be overlaid on the video clicking on the ‘Closed Caption’ icon in the meeting task bar will give the option to show full transcript in the sidebar.
Q: If a presenter mutes themselves in Zoom will transcriptions continue?
A: Yes! This is something users must be cognizant of “The Real Time Transcription for Zoom” app is listening to the users microphone via a web browser connection; muting in the Zoom desktop client will not mute the browser connection to the microphone, which means transcriptions will continue sending.
Q: How do students download their lecture transcripts?
A: The instructor must enable this functionality. Once enabled, they will click the “Save Transcript” at the end of recording. Faculty are also then able to add the transcript to their institution LMS for historical reference.
Q: How much does it cost to run?
A: Cost breakdown is as follows, this works out to less than $1.50/hr to run:
Amazon Transcribe. $0.0004 per second. Usage is billed in one-second increments
https://aws.amazon.com/transcribe/pricing/
AWS API Gateway:
REST API – First 333 million requests $3.50
HTTP API – First 300 million requests $1.11
https://aws.amazon.com/api-gateway/pricing/
AWS Lambda:
Requests $0.20 per 1M requests
Q: If you want the lecture transcriptions to be available to students how would you do this?
A: One option would be to save the transcripts and upload to the course LMS.
Visuals
About the University of British Columbia Cloud Innovation Centre (UBC CIC)
The UBC CIC is a public-private collaboration between UBC and Amazon Web Services (AWS). A CIC identifies digital transformation challenges, the problems or opportunities that matter to the community, and provides subject matter expertise and CIC leadership.
Using Amazon’s innovation methodology, dedicated UBC and AWS CIC staff work with students, staff and faculty, as well as community, government or not-for-profit organizations to define challenges, to engage with subject matter experts, to identify a solution, and to build a Proof of Concept (PoC). Through co-op and work-integrated learning, students also have an opportunity to learn new skills which they will later be able to apply in the workforce.
