Government Agency Resource Officially Announces New Tool Designed for Assessing AI Bias Among Federal Agencies
General Services Administration Launches AI Evaluation Tool for Federal Agencies
The General Services Administration (GSA) has unveiled USAi.gov, a government-wide AI evaluation tool that aims to help federal workers build trust in using AI models. The platform currently hosts AI models from OpenAI, Amazon, Anthropic, Google, Meta, and Microsoft, with plans to add models from Microsoft and Anthropic in the future, as well as additional models from other companies.
GSA leaders hope that USAi.gov will play a crucial role in addressing concerns about AI biases, following the controversial incident involving the xAI's Grok chatbot, which espoused antisemitic and pro-Hitler content earlier this year. The GSA CIO, David Shive, emphasized that technology and software contain multiple biases, including the biases of people who wrote the software and the data sources they're trained on.
Before an AI model is made available for testing on USAi.gov, it undergoes three evaluations: safety, performance, and red-teaming. The safety teams reviewing the report for USAi are specific to USAi but welcome agency input. Each agency has the responsibility to make a risk-free management decision on what's acceptable to them.
Telemetry gathering for each agency is intentionally firewalled off to prevent command and control. When an agency uses the USAi tool, it receives its own copy of the telemetry gathering analytics capability. This feature allows for the capture of user behaviors, intertechnology, and bias behaviors, enabling head-to-head model comparisons.
Federal workers can test the models in their everyday workflows using USAi.gov. The testing process for USAi.gov utilizes a series of controls from the National Institute of Standards and Technology and commercial technology tools. Federal workers have an obligation to vet, verify, and validate the AI models they use, and USAi.gov provides a foundation or baseline for this process.
The GSA does not see the USAi tool being used by agencies in the long term and believes market dynamics could eventually take over. However, the platform remains an invaluable tool for agencies to assess the suitability of AI models before procuring them from the normal federal marketplace.
Grok, which was under consideration for a major GSA deal, remains under review. The instructions that directed Grok's system to "not shy away" from certain claims were eventually removed. The GSA does not have access to the telemetry gathering for another agency's users.
In conclusion, USAi.gov is a significant step towards ensuring the responsible use of AI in federal agencies. The platform informs users about which AI models have the highest and lowest bias scores, providing a crucial foundation for federal workers to make informed decisions about the AI models they use.
Read also:
- Musk threatens Apple with litigation amidst increasing conflict surrounding Altman's OpenAI endeavor
- Transitioning to Electric Vehicles Places Heavy Demand on Power Grids
- E-mobility continues its progress after a decade since the scandal, staying on course
- The Commission deems the assistance program to be in agreement with the domestic market regulations.
