Teaching Assistant Bot (TA BOT) for Software Engineering (SE)

When expanding the intake size of a higher education institution, maintaining educational quality requires proportional increases in both physical and human resources. However, augmenting human resources, particularly competent staff, poses greater challenges than expanding physical infrastructure. Insufficient academic staff, including teaching assistants, directly impacts student interaction and engagement, leading to a decline in educational quality. To address this, our project introduces an AI-based Teaching Assistant BOT designed to facilitate student learning through conversational interactions, mitigating the impact of limited human resources. The deployment of such TA BOTs is anticipated to alleviate the need for an excessive number of teaching assistants in high-enrollment academic courses.
Students commonly encounter difficulties comprehending new subjects, necessitating one-on-one support to prevent loss of motivation and disengagement from the course. In large courses, the traditional ratio of one teaching assistant to 15-25 students limits their ability to provide personalized assistance promptly. The TA BOT serves as a real-time online alternative, offering immediate support tailored to individual student needs.

Recent advancements in Conversational AI tools, exemplified by Chat GPT from OpenAI and Bard from Google, have gained popularity for online subject exploration. However, these tools rely on publicly available knowledge on the web, providing answers in a context-independent manner and often yielding different responses to the same question. Hence, these tools could provide limited assistance to students since they do not depend on the course content, syllabi, or references in the subject when answering questions in a particular educational context. In contrast, the TA BOT leverages a Large Language Model (LLM), designed to infer course resources as primary references, ensuring contextually relevant responses. By aligning with course content, syllabi, and references, the TA BOT minimizes inaccuracies and guides students along the correct learning path.

Development of the TA BOT utilized 'gpt-3.5-turbo,' a potent LLM from OpenAI, fine-tuned for conversation using the using course resources of a Software Engineering course. Employing the Retrieval Augmented Generation (RAG) methodology, supported by the LlamaIndex framework, we created a comprehensive dataset using the reference book of a Software Engineering course. The TA BOT, adopting the persona of a Teaching Assistant, delivers contextual interactions based on subject matter questions while prohibiting academic offences or malpractices.

Even in cases where the fine-tuned knowledge lacks an answer, the TA BOT can provide responses beyond the dataset, showcasing its versatility. Preliminary tests confirm the TA BOT's ability to furnish precise, high-quality answers to Software Engineering-related queries. Positive student feedback underscores its potential to enhance the educational experience. The tool is publicly accessible for experimentation at "http://chat.ucsc.cmb.ac.lk", offering a promising solution to sustain educational quality amidst resource constraints and escalating student demands in higher education institutes."