BioQA-BERT: A Hybrid Large Language Model for Interactive Malayalam Question Answering System on Health Domain

This study introduces the development of an innovative Question Answering System(QAS) for the Malayalam language in the health domain, aiming to bridge the gap between human language comprehension and machine reasoning, using advanced NLP techniques, transformer-based models and Large Language Models. This research presents a sophisticated approach to understanding and responding to health-related queries in the Malayalam language. Trained different BERT models ALBERT, DistilBERT, StructBERT, and RoBERTa with Malayal health data set and then Fine-tuned the models with the MQuAD (Malayalam Question Answering Data set) and then compared the performance of different BERT models. Subsequently, proposed a hybrid LLM- BioQA-BERT by integrating LLaMA with BERT model RoBERTa. The proposed model demonstrates a notable increase in the F1 score of 0.9028, indicating an enhanced Question Answering performance. Despite the inherent linguistic complexities of the Malayalam language and the current state of research in this field, this study makes substantial progress towards effective man-machine communication in the health sector, offering a solid foundation for future advancements in Malayalam QAS.