Using AI to Help Answer Student Questions
Project phases
- Phase 1 - Student Advising Assistant
- Phase 2 - Using AI to Help Answer Student Questions (current page)
Leveraging advancements in technology and generative AI, the Science Advising Office collaborated with UBC Cloud Innovation Centre (CIC) to enhance its AskCali prototype, an AI assistant designed to support students with academic advising questions. The improved prototype performs better and more accurate responses while being cost effective to support students with academic guidance. Ultimately helping students with decision-making to advance their studies.
Interpreting the complexities of academic requirements can be a challenge for any student. To address this, the first phase of the project introduced AskCali, a tool designed to provide 24/7 advising support using conversational interface, offering an easier way to interact with the information.
Approach
This project phase implemented new AI technology to provide more accurate responses and improve the performance of the Student Advising Assistant, leveraging advancements in the field of generative AI.
Innovation
The project combines a student’s questions and contextual information with related documents from university sources, including the UBC Calendar and Faculty of Science web content, to provide accurate and helpful answers. The web app prompts students to provide their academic context before their question. By combining students’ inputs, the system is able to enhance the information returned and using generative AI improves the delivery of academic advice, making it easier for students to navigate their academic journey.
Screenshots of UI
The Main Page
A student wanting to ask a question to an advisor can access the Student Advising Assistant from the web.
If this is a user’s first time using the Student Advising Assistant, there is information provided such as “About” to learn more about the solution or “Instructions and Suggestions” to help them navigate.
Asking a Question
To get help from the solution, the user can enter information on their Faculty, Program or other information. The more detail entered, the more refined the prompt is for the LLM.
If the user has a question, they can enter it in the question box and relevant information in the box above.
Receiving a Response
![Student Advising Assistant response view: "The Faculty of Applied Science at UBC offers a wide range of specializations (also know as majors for undergraduate students. Here are the specializations offered: [list of specializations]".](https://sandbox-cic-2021.sites.olt.ubc.ca/files/2024/09/image-5.png)
The user interface retrieves and displays information from the LLM. Depending on whether the information was helpful or unclear, the user can provide feedback that can be used in the backend to improve the AI response.
If desired, the user can submit feedback regarding the response and references.
Viewing References
The solution will produce a list of references pertaining to the response.
The user can view details about the references, including relevant links.
Technical Details
The solution is built around two core components: a data ingestion pipeline and a question-answering system. The data ingestion process, orchestrated through Amazon Elastic Container Service (Amazon ECS), scrapes and processes content from specified university websites, preparing it for use in the question-answering system.
For answering questions, the system leverages Amazon Bedrock to host the Llama 3 8B model, providing scalable, serverless access to advanced AI. Using PostgreSQL and the pg_vector extension within Amazon Relational Database Service (Amazon RDS) as the vector store, the system efficiently retrieves relevant documents, combines them with the user’s question, and delivers precise, contextually relevant answers.
Architecture Diagram
Step-By-Step
Expand the sections below to learn more about the architecture diagram in-detail.
Text Generation
User Interaction and Query Processing
A student uses a web app hosted on Amazon Elastic Beanstalk to submit a query. The app uses semantic search to find related documents in an Amazon RDS PostgreSQL database. It then queries a model on Amazon Bedrock with these documents as context to generate a response, which is displayed to the user along with the reference documents.
Logging and Feedback
The system logs all queries and responses by requesting an Amazon Lambda Function to store this data in the Amazon RDS PostgreSQL database. User feedback is also stored this way, helping to improve the system.
Lambda Functions
User Management
During the Inference Stack, an Amazon Lambda Function creates a database user with limited privileges, ensuring security. The function retrieves credentials from Amazon Secret Manager to create the user, who can manage data without admin rights.
Table Management
Another Lambda function ensures the creation of feedback, logging, and update tables if they do not exist, maintaining the necessary database structure.
Data Ingestion
Fetching Feedback and Logs
Administrators can run a Lambda function via the Amazon Console to retrieve logs and feedback from the database, format them into CSV files, and store them in an S3 Bucket.
Web Scraping and Data Processing
Admins can configure data processing settings by updating a configuration file in the S3 Bucket. This triggers an ECS task to scrape websites, process data, compute vector embeddings, and store these embeddings in the Amazon RDS PostgreSQL database.
Container Management
Amazon Elastic Container Registry (ECR) integrates with Amazon Elastic Container Service (ECS) to manage and deploy Docker container images. ECS automatically pulls the latest images from ECR, ensuring smooth updates and efficient management of containerized applications
Learn more about the open-source prototype on GitHub.
Video
Acknowledgements
Photo by Sergey Zolkin on Unsplash.
About the University of British Columbia Cloud Innovation Centre (UBC CIC)
The UBC CIC is a public-private collaboration between UBC and Amazon Web Services (AWS). A CIC identifies digital transformation challenges, the problems or opportunities that matter to the community, and provides subject matter expertise and CIC leadership.
Using Amazon’s innovation methodology, dedicated UBC and AWS CIC staff work with students, staff and faculty, as well as community, government or not-for-profit organizations to define challenges, to engage with subject matter experts, to identify a solution, and to build a Proof of Concept (PoC). Through co-op and work-integrated learning, students also have an opportunity to learn new skills which they will later be able to apply in the workforce.
