SCORE (Structured Clinical Output Rating Engine)
Description
SCORE (Structured Clinical Output Rating Engine) uses o3-mini LLM to grade the quality of clinical notes that have either been created by a human or AI and provides feedback to the user. When a transcript of the patient encounter is available, SCORE employs embedding-based semantic analysis to enhance patient safety and documentation accuracy. Embeddings convert text into numerical vectors that capture meaning, enabling the system to measure how semantically similar—or different—statements are between the clinical note and the source transcript. This capability powers two critical safety detectors: Hallucination Detection identifies factual-sounding claims in the note that lack support in the transcript. This is essential for AI-generated documentation, where models may produce plausible but ungrounded statements about medications, allergies, diagnoses, or procedures. Claims with weak transcript support are flagged with risk levels (high/medium/low) based on their potential clinical impact. Contradiction Detection finds instances where the note and transcript discuss the same clinical topic but present conflicting information—such as differing medication dosages, mismatched vital signs, or inverted symptom reports (e.g., "denies chest pain" versus "reports chest pain").
Detailed example
Output: Quality Score & Actionable Guidance Returns a hybrid quality score (1.0–5.0) and letter grade computed from three weighted components: PDQI-9 Analysis (70%) – AI-driven evaluation of 9 clinical documentation dimensions (up-to-date, accurate, thorough, useful, organized, concise, consistent, complete, actionable) Heuristic Metrics (20%) – Rule-based assessment of length, structure, and redundancy Factuality Verification (10%) – Claim validation against source transcript (when available) Each dimension includes narrative feedback, supporting evidence excerpts from the note, and specific improvement suggestions to guide documentation enhancement.
AI / analytics pattern
Generative AI: AI that generates new or synthetic content (e.g., images, videos, audio, text, code).
Automation level / stage
a) Pre-deployment – The use case is in a development or acquisition status.
Expected benefit
Improve clinical documentation quality through standardized evaluation and scoring and create efficiency for documentation audit staff.
Controls / human review
ATO: Not reported; PIA: Not published