The NEF DSI/AI project addresses local validation of AI-based clinical decision support (CDS)/decision support interventions (DSI) in provider settings.
Description
Assess quality of AI-based clinical decision support (CDS) tools
Detailed example
Outputs are not generated by AI, but rather use open source information to assess outputs from other AI-based tools. This tool's outputs are metrics that measure, for example, accuracy and precision of external AI predictions of disease onsets in local patient populations.
AI / analytics pattern
Other
Automation level / stage
a) Pre-deployment – The use case is in a development or acquisition status.
Expected benefit
[LAVA, or "Local AI Evaluator", does not itself use AI.] LAVA would assist clinicians in assessing the accuracy, and therefore usefulness, of AI diagnosis tools. AI diagnosis tools are developed to apply to a national population, rather than to smaller, local populations, such as those served by small providers with one or few physical locations. These smaller, local patient populations may have different demographics than that on which the AI-based tool was trained, so the LAVA tool can help illuminate these differences and how the AI tool may apply to the local population. This can help providers learn how to best use their AI diagnosis tools.
Controls / human review
ATO: Not reported; PIA: Not published