Working to reduce risks from transformative AI.

At CARMA, we envision a future where humanity retains control over AI’s transformative impact. We work toward a world where advanced AI is developed with foresight, strong governance, and proactive safety measures.

CARMA’s mission is to lower the risks to society and the environment from transformative AI through research and advocacy of that research.

CARMA's approach blends systems risk management, policy research, and technical safety to foster a  "defense in depth" suite of tools, policies, and best practices to manage risks from the most potent AI systems. We consider risks to society, individual well being, and the environment as primary concerns. Our team draws from diverse backgrounds in computer science, risk management, complexity theory, international relations, security studies, and public policy to identify, and facilitate others’ identification of, critical systemic vulnerabilities and to develop layered protections against such AI risks. We recognize that no single solution will suffice - instead, we pursue a complimentary set of technical, governance, and security measures designed to work in concert. From evaluating novel risk containment strategies to strengthening institutional guardrails and building societal resilience, our work aims to create multiple layers of safety and security.