Multilingual AI Safety Evaluations

AI is being deployed in classrooms, clinics, welfare systems, and crisis hotlines, promising to transform how people access essential services. But as these systems expand globally and reach multilingual users, a critical gap remains: how they perform across languages. Most AI evaluations still focus on English, leaving other languages with weaker safeguards and limited testing. The result is inconsistent responses that undermine trust, safety, and human rights.

The Multilingual AI Safety Evaluation Lab is an open-source platform to uncover and address these risks. By enabling side-by-side comparisons of English and non-English outputs, the platform helps evaluators identify and document inconsistencies in usefulness, factual accuracy, privacy, accessibility, censorship, and fairness. It combines human and AI evaluators (LLM-as-a-Judge) to highlight where their judgments align or diverge, emphasizing the context and lived experiences of multilingual communities.

The Lab equips researchers, civil society organizations, and AI developers with tools to assess, document, benchmark, and mitigate AI safety risks across languages.