Risk identification and mitigation, lifecycle of impact assessments, documentation and transparency, auditing and compliance, precautionary measures, and licensing

Example questions: 

• Does the bill require covered entities  to conduct Impact/Risk Assessment or similar evaluations?
• Does the bill require maintenance of Impact/Risk Assessment documentation?
• Does the bill specify or otherwise acknowledge risks arising from development by requiring testing the AI tool/system for validity?

Privacy rights, data sensitivity, collection and minimization, usage and retention, transfer and sharing, deletion, security, and data subject rights

Example questions: 

• Does the bill require organizations to document the purposes for which personal data is collected, used, processed or retained?
• Does the bill require explicit, informed consent from individuals before collecting their personal data?

 Impact and mitigation practices to prevent discriminatory outcomes

Example questions: 

• Does the bill explicitly include legally protected characteristics (e.g., race, gender, age, religion, disability) in its definition of discrimination or bias?
• Does the bill define "algorithmic discrimination" (or a similar term) to characterize unfair treatment toward specific groups?

Job displacement, upskilling initiatives, and industry-government collaboration

Example questions: 

• Does the bill call for the analysis of challenges faced by workers affected by automation or AI implementation?
• Does the bill call for the analysis of demographics that may be most vulnerable to AI displacement?

Development of new institutions for governance, an elaboration of interagency collaboration, and mechanisms for enforcement

Example questions: 

• Does the bill mandate the establishment of a new entity?
• Does the bill outline clear, measurable objectives for the new entity that must be achieved within defined timelines?

• We found that 95% of the policies contained definitions for AI or ADS, and 83% of bills were found to be highly similar to established existing AI definitions. On the other hand, definitions of concepts related to generative AI were only found in 30% of bills. Only one bill provided a quantitative resource-based definition of generative or foundation models.

• Many bills addressed matters regarding Accountability & Transparency,  Bias & Discrimination and Institution. Specifically, when we restricted bills that scored at least 50% of Y/N questions per category, we found 10 out of 23 bills that covered each of these categories. On the other hand, while many bills did have some components in Data Protection, only one had at least 50% coverage. Note that these bills are typically AI/ADS-related bills, where many states have separate bills to address data privacy specifically.  Lastly, there was a general lack of coverage for Labor Force matters across these bills.

• We found that none of the bills analyzed have comprehensive coverage across all categories. Specifically, we counted the number of categories with high coverage, i.e. at least 50% of “Yes” per category. Amongst the 23 bills analyzed, no bill covered all 6 categories highly.
• As Data Protection and Labor Force were not highly covered in these AI/ADS bills, we also analyzed comprehensive coverage without these categories. With the exclusion of Data Protection, we found only one bill highly scored across the remaining 5 categories. And with the exclusion of both Data Protection and Labor Force, there were 5/23 bills having high coverage for the remaining 4 categories.
• By analyzing the number of categories highly covered by a bill, we found differences between federal and state bills. On average, it appears state bills tended to highly cover more categories than federal bills. Only 14.2% of the federal bills covered at least half of the categories, while 68.8% of the analyzed state bills covered similarly.

• We compared the legislative texts to the model bills in two ways. The first way compared the overlap between elements that exist between a given legislative bill and model bill. In other words, we calculated the percentage of similar “Yes” items over the union of all “Yes” items between the two bills. By doing so, we found that many bills were more similar to one model bill than the others.

We acknowledge financial support from the Media and Democracy Fund for this work, and also acknowledge the generous access to legislative data provided to us by Plural Policy. We would like to thank a number of external experts who provided invaluable feedback during the early stages of developing this framework, as well as a number of scorers who helped us shape the framework and score different bills. 

While developing the framework, key team members left to pursue other opportunities. Sasa Jovanovic was one of the primary leaders of this project in its early stages of development, and continues to advise us. Rachel Kim helped develop parts of the framework before graduating.