Using Amazon Chime and SIP Media Applications to improve emergency communications in rural areas

Using Amazon Chime and SIP Media Applications to improve emergency communications in rural areas

A view of some green and snowy mountains.

By Trevor Flanigan

Working on STARS was a great learning experience for me. It was the first project that I built using AWS, and I had so much support from the members of the FLASH and Chime teams, as well as fellow students and managers at the UBC Cloud Innovation Centre (CIC). My experience at the CIC has been entirely positive, and it has really helped me improve as a developer. In this blog post, you will read about the work done on the Virtual Communication for Remote First Responders project done with the Shock Trauma Air Rescue Service (STARS) team.

Definitions

  • Emergency Support Center: A communications hub that hires Agents who are highly trained in emergency communication.
  • (Emergency) Responder: A professional who is at the scene of an emergency. i.e. Firefighters, Police, Medics, or another person who provides aid.
  • (Medical) Specialist: A professional with expertise in a specific medical field, from whom emergency responders can request remote assistance.
  • (Service Desk) Agent: An employee of an emergency support center in charge of managing, recording, and recordkeeping of an active call.
  • SIP Media Applications (SMA): An Amazon Chime feature that allows a lambda function to receive input from a telephone caller. It allows developers to capture audio and keypad inputs, as well as play audio to the caller.

The Problem

Emergency responders in rural and remote areas are essentially ‘alone’ and find it difficult to access expertise during a time sensitive medical emergency. Currently, an individual at the scene of a medical emergency dials 911. During the call, the 911 dispatcher would collect essential information and determine if the call should be transferred to an emergency support center; however this information is not always effectively transmitted in the process. The resulting lack of information would hinder the first responder from rapidly connecting with the right specialists and experts based on the requirements of the situation. 

The Solution(s)

The project’s team, with members from the University of British Columbia Cloud Innovation Centre (UBC-CIC), and the Amazon FLASH Team introduced a set of solutions that aim to streamline the medical assistance process: First Responder*, a mobile communications application for both emergency responders and specialists, and Service Desk*, a dashboard where an agent can monitor and connect responders to relevant specialists.

First Responder

First Responder is an application that is designed to be adaptable to the changes in connectivity in rural areas. It supports two methods of connection: telephony and internet, with telephony acting as a fallback in the absence of internet. The base application provides voice connectivity over either telephone or internet, with additional features being provided while connected to the internet. The advanced features include video communication and the ability to upload photos, so medical advice can be provided as if experts were on the scene. Users can create profiles with information about their areas of expertise.  

The emergency responder Home Screen that has a "call service desk over data", "phone service desk" and "edit profile" options.
Emergency Responder Home Screen
The internet call interface that has a list of callers present, the mute, speaker, video, notes and leave call buttons.
Internet Call Interface
Example of sending a photo in the chat during a call.
Chat with Image Upload
An image of an emergency responder's profile, where the phone number is listed and first name, last name, and occupation can be modified.
Emergency Responder Profile

Service Desk

The Service Desk web application is a dashboard for managing an ongoing stream of meetings. The Service Desk requires an internet connection to run, but it is able to list calls made over both telephone and internet, using an SMA. 

Agents are in charge of joining new meetings and assigning appropriate specialists to each situation. In addition to the dashboard view, attendants have the option to switch to a map view, powered by Amazon Location Service. When a meeting is created using the internet, the location of the meeting is recorded and displayed as a pin drop on the map. This allows the specialist and service desk team to more precisely locate an ongoing call. Geolocation capabilities can be especially helpful in rural areas (albeit those which have internet and/or LTE), where it can be difficult to describe one’s whereabouts. 

The image of the Service Desk shows a list of meetings and details including the meeting ID, title, status, attendees, action, start time, source duration, and service desk attendee.
Service Desk Dashboard
The service desk map is a map that has pins in where calls are located and originated from.
Service Desk Map
The UI of the service desk has a view of the attendees from their cameras, list of attendees and the call's chat.
Service Desk Call UI

Architecture

The UBC-CIC team opted to use React with AWS Amplify libraries for the web app, with Amazon Cognito for authentication, Amazon S3 to store any uploaded files, and Amazon DynamoDB as the data layer.  As the application is interconnected with the Public Switched Telephone Network (PSTN), user phone numbers must be collected and verified. This was accomplished using Cognito Custom Triggers, which send a user a one time login passcode via SMS. The solution also uses Amazon Location Service for mapping and geocoding. 

Architecture Diagram

The architecture diagram illustrates the flow of the application using numbered icons of AWS services.

User Flows

Internet Connected Flow

  1. A specialist registers (0) with the app and is ready to be alerted to join a conference when their expertise is required. First Responders already have the app downloaded, and have verified their phone number (0) (or will have internet connection at the scene of the emergency).
  2. Later, an emergency responder requires a specialist’s assistance. They call an Emergency Support Center using their phone’s data. (1 in Architecture Diagram). Their phone number is used to create a Meeting Item in the DynamoDB (3) with their user profile. The responder is put into an empty call.
  3. The Service Desk (12) receives a notification, and the meeting appears on the dashboard. The agent joins the call using the JoinMeeting Lambda (4). The responder tells the agent the situation, and the agent selects a relevant specialist to notify (7,8) from the SpecialistProfile table (3). 
  4. Specialist receives an SMS (7). It redirects them to First Responder. The landing screen points them to join the meeting (4). 
  5. At this point, the specialist and responder are connected. They have voice/video chat via Chime (4) and can send photos (6). Once the meeting is ended, a DynamoDB Trigger (3) sets the meeting to end automatically. It is removed from the service desk dashboard (12) and the specialist’s notifications (1).

Telephony Flow

  1. A specialist registers (0) with the app and is ready to be alerted to join a conference when their expertise is required. First Responders already have the app downloaded, and have verified their phone number (0).
  2. Later, an emergency responder requires a specialist’s assistance. They call an Emergency Support Center using the buttons in the First Responder App (1). This redirects them to their phone’s native phone call screen. SMA (5) automatically uses their phone number to create a Meeting Item in the DynamoDB (3) with their user profile. The responder is put into an empty call. 
  3. The Service Desk (12) receives a notification, and the meeting appears on the dashboard. The agent joins the call using the JoinMeeting Lambda (4). The responder tells the agent the situation, and the agent selects a relevant specialist to notify (7,8) from the SpecialistProfile table (3). 
  4. Specialist receives an SMS (7). It provides them with a phone number to connect to an SMA for joining the meeting (5). Clicking the provided phone number takes the specialist to their phone’s native call screen.
  5. At this point, the specialist and responder are connected. They have voice chat via Chime (4). Once the meeting has ended, a DynamoDB Trigger (3) sets the meeting to end automatically. It is removed from the service desk dashboard (12) and the specialist’s notifications (1).

Notes

These methods can be mixed and matched, depending on each user’s connectivity method. For example, a meeting created over the internet can be joined by a specialist over telephony, and vice-versa. The only user that requires internet connection at the time of the call is the Service Desk Agent. 

Both Specialist and Emergency Responder are required to have an internet connection in order to download and verify their phone numbers. This only needs to be done once, then again whenever the user chooses to log out. 

Expansion Beyond the Prototype Phase

If the prototype is to be expanded upon, this app’s PSTN system should connect to an emergency response center’s existing PSTN system as opposed to existing in the cloud.  Because PSTN is more reliable than voice over internet, in a future build, voice should be over PSTN at all times, even during the internet-connected flow.

With even further support, the offline capabilities of the application can be improved using satellite technology. In a perfect version of the application, it is accessible from any point on the planet, regardless of cellular and internet connectivity (via satellite). 

“Scenes from The University of British Columbia (Vancouver BC, Canada)” by @CarShowShooter is licensed under CC BY-NC-SA 2.0