Evaluating LLM Reliability for Reproductive Health Information
When people turn to AI tools for everyday health advice, getting accurate, unbiased information isn't just convenient, it's absolutely critical. This project, a collaboration between The Re/Imagination Lab and the Center for Technological Responsibility, Re-imagination, and Redesign (CNTR), evaluates how popular AI chatbots handle sensitive, fast-changing questions around reproductive health and contraception.
Because state laws and access to healthcare vary widely across the country, AI models often respond differently depending on where a question is asked or how it's phrased. By testing leading AI platforms (ChatGPT, Claude, Gemini, and Perplexity) in both English and Spanish across various legal environments, the project measures whether these tools provide accurate medical information and fair assessments of legal risks.
Ultimately, the goal is to bring transparency and accountability to AI in public health. By tracking these tools over time, the initiative highlights where AI succeeds, where it falls short, and how it can be improved to ensure everyone gets trustworthy health information regardless of where they live or what language they speak.
Project Team
-
Suresh Venkatasubramanian
Director of the Center for Technological Responsibility, Reimagination, and Redesign, Professor of Data Science and Computer Science -
Harini Suresh
Assistant Professor of Computer Science -
Michelle L. Ding
PhD Student in Computer Science, Founder of SRC Handbook, AI Expert for CNTR SciToons -
Haya Bugshan
Undergraduate Student in Computational Neuroscience & Philosophy