Establish localized Large Language Model (LLM)
Description
AI Enablers (Enterprise solutions that enable AI to integrate into IT services infrastructures)
Detailed example
AI-Powered Service Performance Metrics. Efficiency and Responsiveness: Track reductions in response times versus the pre-LLM baseline, first-contact resolution without escalation, and inquiry deflection handled entirely by the LLM. Accuracy and Consistency: Evaluate response accuracy through audits against official documentation, measure alignment with current Fiscal Service policies, and track reductions in incorrect or inconsistent responses compared to manual baselines. Security and Compliance: Ensure zero unauthorized access through effective role-based controls, conduct quarterly reviews for compliance with federal privacy, cybersecurity, and records management standards, and pass all required internal and external security audits. User Adoption and Satisfaction: Measure internal adoption rates, collect user satisfaction feedback through surveys or ratings, and assess training effectiveness using feedback and usage metrics. Operational Impact: Quantify staff hours reallocated from repetitive inquiries to higher-value work, track operational cost savings, and measure the breadth of internal systems and processes covered by the LLM’s knowledge base.
AI / analytics pattern
Generative AI: AI that generates new or synthetic content (e.g., images, videos, audio, text, code).
Automation level / stage
b) Pilot – The use case has been deployed in a limited test or pilot capacity.
Expected benefit
Consolidation of multiple research initiatives into an aligned approach to research an enterprise LLM that will be trained on approved internal documentation to provide accurate, consistent, and rapid responses to employee inquiries. This will improve service delivery, reduce case escalations, and free personnel to focus on complex, mission-critical work
Audit / financial statement impact
The tool, once deployed, is only for productivity and does not impact any of the defined high-impact areas
Controls / human review
ATO: No; PIA: Not published
Data needed
N/A. Evaluating tools only at this stage