Dynamic Domain Knowledge Injection In Large Language Models Using Context-Gated Prompt Engineering

Abstract

Large Language Models (LLMs) have excelled in various NLP tasks, such as natural language understanding, question answering, and knowledge-intensive applications. But it is still difficult to be used in specialized fields, such as biomedicine, because the knowledge acquired in pre-training can be incomplete and outdated, and it may lack the knowledge for specific queries in this field. To include external knowledge, current methods typically rely on fine tuning or prompt engineering or Retrieval-Augmented Generation (RAG). Conventional RAG can include irrelevant or extraneous context in prompts, impacting the accuracy of responses and leading to higher token usage, and fine-tuning is time-consuming and demands significant computing power, with the need to update the model repeatedly. To overcome these challenges, this study introduces Context-Gated Knowledge Injection LLM (CGKI-LLM), a fine-tuning-free domain-specific customization framework. The proposed method decomposes a user query, fetches candidate knowledge, computes the semantic relevance of candidate knowledge, applies a context-gating process to filter out knowledge not related to the user query, and then generates the final prompt. PubMedQA is the main biomedical question-answering set on which experiments are conducted. The proposed framework is compared with a base LLM, conventional prompt engineering, and standard RAG in terms of accuracy, precision, recall, F1 score, hallucination rate and context-related measures. Results demonstrate that the use of CGKI-LLM with standard RAG results in 89.20% accuracy and an F1 score of 88.92% whereas using standard RAG only, the accuracy rate is 84.30%. CGKI-LLM decreases the hallucination rate from 7.20% (standard RAG) to 3.80%. These results show the effectiveness of selective context gating for reducing irrelevant knowledge injection and enhancing domain-specific question answering. The proposed framework offers a practical solution for integrating specialized knowledge into LLMs without altering the model's parameters.