Assessing Large Language Models for Synthetic Survey Data Generation
Description
Survey data de-identification is crucial for the NCHS to maximize data utility while protecting privacy, but determining and applying modern best practices requires further research. NCHS conducts national surveys and releases microdata (data containing information about individuals) for public use. To protect survey participants’ confidentiality, statistical disclosure limitation techniques have been used to de-identify data, but these methods have drawbacks of losing statistical properties of the original data and thus limiting useful analyses. Additionally, these methods are not designed for very large data or text data. Use of synthetic data may offer another option. The goal of synthetic data is to preserve essential statistical features and variable relationships of the original data such that statistical inference based on the synthetic data is close to that of the original data. Large language models (LLMs) may be able to address limitations of statistical methods for synthetic data creation, especially for natural language data. We aim to advance knowledge of this application of LLMs to enable staff to select the optimal tools for synthetic data generation.
Detailed example
Continuous, categorical, and free text data that matches properties of original survey data.
AI / analytics pattern
Generative AI: AI that generates new or synthetic content (e.g., images, videos, audio, text, code).
Automation level / stage
a) Pre-deployment – The use case is in a development or acquisition status.
Expected benefit
Current statistical methods for synthetic data generation have drawbacks such as difficulty handling very large datasets, steep learning curve for people with less statistics or coding background, and inability to generate natural language data. Thus, if LLMs are evaluated to be successful at synthetic survey data generation, this alternative method would enable more data synthesis at scale, more data synthesis that can be conducted by staff with various levels of statistics backgrounds, and the first ever release of synthetic survey text data.
Controls / human review
ATO: Not reported; PIA: Not published