Sample Refinement: Frame API
Description
BLS establishment-based surveys, such as the Survey of Occupational Injuries and Illnesses (SOII), the National Compensation Survey (NCS), and the Occupational Requirements Survey (ORS), use the Longitudinal Database (LDB) as the frame for their survey samples. For each of these programs, there is a significant time-lag between when the sample is drawn for a program and when data collection begins. A critical and time-consuming part of data collection is sample refinement: checking the latest Quarterly Census of Employment and Wages (QCEW) and LDB data to adjust for any changes to the sampled establishment during that time-lag. The QCEW/LDB Frame API allows fast retrieval of the latest QCEW/LDB data for a nationwide sample so that users can perform sample refinement more efficiently. The SOII program uses the Frame API to perform over 20 sample refinement comparison checks, which outputs a report for the data collectors. The Frame API uses TF-IDF vectors and cosine similarity to compare company names, mailing addresses, and unit descriptions between the survey sample and the latest QCEW/LDB data to determine what components may have changed. The Frame API also has the flexibility for the users to match records using other combinations of variables.
Detailed example
Similarity score between records
AI / analytics pattern
Classical/Predictive Machine Learning: Models trained on data to make predictions or classifications based on identified patterns or relationships.
Automation level / stage
c) Deployed – The use case is being actively authorized or utilized to support the functions or mission of an agency.
Expected benefit
Efficiency and data quality improvements
Audit / financial statement impact
Used for survey processing for statistical purposes
Controls / human review
ATO: Yes; PIA: N/A
Data needed
SOII and QCEW records