Large Language Model (LLM) Guided Data Dictionary Generation
Description
This LLM-generated data dictionary eases the burden of metadata documentation on the data stewards when integrating their data into FEMADex by creating field definitions.
Detailed example
This LLM model utilizes Retrieval Augmented Generation (RAG) technique for data dictionary generation by first retrieving relevant provided metadata from 1) the source system intake form and 2) an acronym key. The LLM then uses this context to generate clear, brief dscriptions for each field.
AI / analytics pattern
Generative AI: AI that generates new or synthetic content (e.g., images, videos, audio, text, code).
Automation level / stage
b) Pilot – The use case has been deployed in a limited test or pilot capacity.
Expected benefit
By utilizing this LLM-generated data dictionary, it automatically creates field definitions, saving data stewards significant time and effort during data onboarding.
Controls / human review
ATO: No; PIA: Not published
Data needed
This model utilizes metadata documentation provided by datastewards, known as the Source System Intake Form (SSIF), during source system intake for FEMADex. Additionally an acronym key for each source system is also used and provided by the respective data stewards. The SSIF and acronym key are unique for each source system.