The Challenge

Clinicians including Physiatrists and Physiotherapists currently rely on on-premise measurements and patient-reported data to assess the mobility level of their patients recovering from various conditions and injuries affecting their function. Recalling details on physical activity can be a complex cognitive task, and becomes more challenging when some of the patients belong to the older adult population, which makes them prone to have particular memory and recall skills limitations that may affect the effectiveness of interventions designed to increase balance level. Clinicians seek a more effective recovery plan for their patients.

Approach

The solution is designed to support two user types: clinicians and patients. The UBC CIC developed a web dashboard and a mobile app to provide each of the users identified above with the infrastructure to access, view, and monitor test data. The solution supports the clinician’s use of the Berg Balance Scale, a 14 item measurement that assesses a patient’s static balance and fall risk. For the prototype, the Sit-to Stand component was selected. 

The mobile app, which is equipped with inertial measurement unit (IMU) sensors, will capture data on the speed, angular velocity, and orientation of their body using a combination of tools such as an accelerometer, gyroscope, and the magnetometer. The app also allows patients to view past test events and analytics, including a score tracking graph, IMU range graphs, and IMU data graphs from every individual test event. The dashboard will allow clinicians to access the information captured via the patient app, view and download data graphs, and allow clinicians to assign new tests and movement scores to their patients.  The data captured will be used to train a machine learning (ML) model to aid in clinical predictions of future risk probability and patient outcomes. 

Supporting Artifacts

Architecture Diagram

Demo Video

USER INTERFACE

Clinician Dashboard

The Dashboard Homepage

Patient Test Details Page on the Dashboard

Patient Past Test Event Page on the Dashboard 

Patient App

Recording Function on the Patient App

Test Details and Analytics on the Patient App

Clinic App

Technical Details

The solution involves three major sections: (i) clinician dashboard, (ii) mobile app, and (iii) Machine Learning (ML) model. All three components are built to work together to deliver the developed solution. The Application Flow is used by both the dashboard and the mobile app for displaying data, while the Data Input Flow is only initiated in the mobile app.

Application Flow

As part of the authentication process, an administrator uses AWS Cognito to assign other dashboard users to the care provider group for them to have access to balance assessments. 

A Lambda function checks if the user is authorized to access other patients’ balance assessments. If the request is accepted, Appsync calls Lambda resolvers, which retrieves data from Amazon RDS (PostgreSQL), Athena, or S3 to display on the dashboard. Finally, AppSync will return the response from the Lambda resolvers to the application, and display it on the user interface in the form of graphs, giving insights on score trends, measurement range trend, and individual sensor measurement fluctuations.

Data Input Flow

After the patient submits a recording from the app, the app prepares a JSON file and sends it to an S3 bucket. This delivery of JSON files will trigger a Lambda function, which receives the event information and gets the JSON file from the bucket. The Lambda function code, which is made using a Docker image, will first convert the JSON data into CSV and parquet files using the Pandas library, and then store the JSON, CSV, and parquet files in three different folders of the same S3 bucket. The JSON folder is for patient test organization purposes.

Next, the Lambda function checks for a model endpoint.  

1. If the endpoint does not exist: A training job will be generated to train a multi-model. Afterwards, an endpoint will be created. To start a new training job, a minimum of 10 recordings per individual will need to be submitted from a care provider’s account.
2. If the endpoint exists: The code will check for an existing model from the patient. A training job will start if there is not a model. 
3. If the user has an existing model and a test recording was submitted: it will invoke the endpoint, prompting the model to assign a score to the recording. Prediction scores will be sent back to the Lambda function. 

When a training job is created or when a model is added to the endpoint, these files will be stored in another S3 bucket. The score received from invoking an endpoint will be saved in the RDS database using SQL query executions.

Machine Learning Model

For each training job, the model gets the specified patient’s training recording files from a S3 bucket. From each file, the timestamps, care provider score, accelerometer, gyroscope, and magnetometer data are collected. The score will be used as “labels”, while the processed accelerometer, gyroscope, and magnetometer data will be used as the “features” for training. 

A custom Long short-term memory (LSTM) model was created to train the data based on the care provider’s assigned score. A model will be trained for each individual patient. After enough data has been collected to train the model, it will be able to generate prediction scores in the absence of a clinician.

Link to solution on GitHub: https://github.com/UBC-CIC/balance-test-app

Acknowledgements

Photo by: Brian Smith, Providence Health Care