Abstract

The purpose of this study is to analyze the use of Large Language Models (LLMs) for the task of question-answering in a medical context. This is vital as healthcare workers are constantly being overworked due to inadequate resources and an overwhelmingly large number of patients. We explore the possibility of Conversational Agents or chatbots, as a tool to answer common medical questions, especially in the context of developing countries. The methodology of this research paper is informed by an analysis of Natural Language Processing, particularly with Neural Networks and Transformers. We design an Artificially Intelligent Conversational Agent using Google’s BERT, Microsoft’s DialoGPT, and Google’s T5 language models. We evaluate these models on the metrics of BLEU score and Perplexity and supplement them with a survey to establish user preference. We also develop a web-based application for users to test the models in a real-world setting. We try to improve the user-preferred model by integrating it with a heuristic-based model and connecting its context to a medical corpus. Then, we discuss the results of our analysis especially concerning its potential use in developing countries. Though our results indicate great potential, we find that our models contain bias and are capable of misinforming users. Finally, the thesis concludes with a discussion on the limits of LLMs and recommendations for making these models inclusive.

Advisor

Bhowmik, Kowshik

Department

Computer Science

Disciplines

Computer Sciences

Keywords

LLMs, chatbots, AI, NLP

Publication Date

2023

Degree Granted

Bachelor of Arts

Document Type

Senior Independent Study Thesis

Download

Share

COinS
 

© Copyright 2023 Bolanle Oladeji