Deep-Learning approaches to develop candidate lists of terms for use in text searches
Description
The intended use is to increase the comprehensiveness of term lists to be used in text-matching approaches to identifying concepts of interest in free-text medical record notes. Text-matching approaches are an efficient method for identifying text of interest in medical notes. The success of text-matching approaches depends on the quality of term lists used to search for a concept of interest. While humans can generate word lists for text-matching approaches, the generated word lists are likely to be limited and biased by the vocabulary, language habits, population exposure, and local dialects of the human generating the list. Deep learning methods can help to augment candidate term lists to help overcome some of these challenges. Once generated these augmented candidate terms are reviewed by subject matter experts for appropriateness and accuracy of the suggested terms for the concept and use case of interest.
Detailed example
The outputs of this use of AI is a candidate list of terms related to a concept of interest. This candidate list is combined with suggestions generated by human experts and then these are reviewed by a subject matter expert to generate a final list of terms for use in text-matching algorithms to extract medical record free text mentions of interest. For the pre-trained Large Language model approach, the input is a query prompting for synonyms of terms related to the concept of interest. The output is the answer provided by the LLM, including the suggested candidate terms and the original terms that were stated in the query. For the word and phrase embedding models, the input is an initial list of terms that are relevant to the concept of interest and the output are the suggested candidate terms, ranked by their statistical similarity to the original terms, based on the transformation of a clinical corpus into vector space or an “embedding model”. Regardless of the type of model used, all suggested candidate terms are reviewed by subject matter experts to ensure quality term lists are used in all text searches.
AI / analytics pattern
Natural Language Processing: AI that processes, interprets, and shares information in human language.
Automation level / stage
c) Deployed – The use case is being actively authorized or utilized to support the functions or mission of an agency.
Expected benefit
Here we (1) query pre-trained large language models (e.g., ChatGPT or Llama) through a user interface that allows prompting (such as the OpenAI API or CodeLlama) and/or (2) query word and phrase “embedding” models (e.g., Word2Vec and Phrase2Vec) to expand candidate term lists for review by subject matter experts. Our team has found both approaches effective for expediting the generation of more comprehensive term to concept mappings for use in text-matching algorithms that increase the accuracy of clinical concept extraction. We expect this use of AI to improve initial brainstorming of terms related to a given concept, resulting in a more sensitive approach to finding information of interest in free-text clinical notes. Experience to date has found that this approach enriches term sets in ways that increase sensitivity for identification of text of interest in clinical TIU notes when used in our CLEVER natural language processing pipeline.
Controls / human review
ATO: Not reported; PIA: Not published