Richard Mallah founded and leads CARMA, where he directs its portfolio of projects in risk assessment, policy strategy, and technical safety. With over twenty years of experience in machine learning and AI, Richard brings deep expertise across AI algorithms research, research management, systems architecture, product management, and strat
Richard Mallah founded and leads CARMA, where he directs its portfolio of projects in risk assessment, policy strategy, and technical safety. With over twenty years of experience in machine learning and AI, Richard brings deep expertise across AI algorithms research, research management, systems architecture, product management, and strategy. Since 2010, Richard has focused on advanced AI safety, maintaining a sense of urgency about societal risks from AGI. He has developed frameworks for understanding pathways to AGI, mapping risk types and intervention paths, formulating theories of change, and creating governance recommendations, while advancing research in scalable safety and computational ethics. In addition to his work at CARMA, Richard serves part-time as the Principal AI Safety Strategist at the Future of Life Institute, which he joined in 2014, where he conducts research, analysis, advocacy, strategy, and field-building regarding technical, strategy and policy aspects of transformative AI safety. Prior to CARMA, Richard co-led the Fairness, Auditing, Transparency, and Externalities of AI Center of Excellence at management consultancy Keystone Strategy, which provided perspective on AI auditing and multibillion-dollar litigations. His experience heading enterprise risk management systems development at BlackRock during the Financial Crisis gave him unique insight into the interplay among systemic tail risk, technology, multiscale foresight, risk reduction, and catalysts for systemic improvement. Richard holds a degree in Intelligent Systems from Columbia University.
Daniel Kroth is a Senior Researcher at CARMA, where he leads the Public Security Policy program. He also contributes to projects across the Offense/Defense Dynamics, Geostrategic Dynamics, and Comprehensive Risk Assessment programs. His prior experience spans international security and technology policy with appointments at Lawrence Liver
Daniel Kroth is a Senior Researcher at CARMA, where he leads the Public Security Policy program. He also contributes to projects across the Offense/Defense Dynamics, Geostrategic Dynamics, and Comprehensive Risk Assessment programs. His prior experience spans international security and technology policy with appointments at Lawrence Livermore National Laboratory, the Special Competitive Studies Project, and the Wilson Center’s Science and Technology Innovation Program. A firm believer in the importance of mentorship and improving accessibility in technology policy and international affairs, he is a founding member of the Next Frontier Seminar, a nonprofit which supports outstanding undergraduate student research in those fields. Daniel remains active in political science research, particularly in nuclear deterrence and international relations methodology. Daniel holds a master’s degree with concentrations in International Security and Technology and Global Affairs from the Fletcher School of Law and Diplomacy, where he wrote his thesis on a theoretical framework for novel nuclear-conventional entanglement risks presented by emerging technologies. While at the Fletcher School, Daniel studied AI policy and cybersecurity at the Harvard Kennedy School of Government. Prior to his graduate studies, he undertook a year of study at Albert-Ludwigs-Universität Freiburg with support from the German Academic Exchange Service (DAAD) and additional study at Sichuan University. He holds a bachelor’s degree from Michigan State University.
Abra Ganz is a Senior Researcher at CARMA, where she leads the Geostrategic Dynamics program. In this role, she combines technical analysis with political pragmatism to analyse how transformative AI (TAI) is changing the dynamics of multilateral competition and cooperation. Abra’s aim is to propose and stress-test mechanisms of internatio
Abra Ganz is a Senior Researcher at CARMA, where she leads the Geostrategic Dynamics program. In this role, she combines technical analysis with political pragmatism to analyse how transformative AI (TAI) is changing the dynamics of multilateral competition and cooperation. Abra’s aim is to propose and stress-test mechanisms of international cooperation using tools drawn game theory, mechanism design, and international relations. Prior to CARMA, Abra worked as a researcher at Yale’sDigital Ethics Center where she focused on how physical infrastructure can be used to govern digital systems. She has also done technical AI safety research at ETH Zürich (on adversarial robustness) and MIT (on inverse reinforcement learning) and authored a chapter on Proxy Gaming in the 'AI Safety, Ethics, and Society' textbook. Abra holds an undergraduate degree in Classics from the University of Oxford and a Master’s in Logic from the Institute of Logic, Language, and Computation at the University of Amsterdam.
Anna Katariina Wisakanto is a researcher and strategist at CARMA, where she leads the Comprehensive Risk Assessment program. In this role, she develops novel analytical methods derived from first principles to model pathways connecting AI capabilities to potential global-scale harms and to identify those posing the greatest risk. Anna com
Anna Katariina Wisakanto is a researcher and strategist at CARMA, where she leads the Comprehensive Risk Assessment program. In this role, she develops novel analytical methods derived from first principles to model pathways connecting AI capabilities to potential global-scale harms and to identify those posing the greatest risk. Anna combines her background in philosophy, engineering physics, and complex adaptive systems to analyze and address risks posed by advanced AI systems. Since entering the AI field in 2018, she has worked across industry, academia, and business contexts, focusing on evaluations, risk assessment, and developing a holistic understanding of AI systems' actual risks and limitations. Anna has also explored the intersection of philosophy of AI, complex adaptive systems, and cognitive science, investigating topics such as the impact of global AI systems on human cognitive and moral autonomy. This multifaceted perspective allows her to bring a unique blend of holistic thinking, creative problem-solving, and ethical considerations to the challenges of managing risks from highly transformative technological systems like AI. Anna holds an Engineering Physics degree from Chalmers University of Technology, where she wrote her thesis on quantum error correction.
Sterlin Waters is a Senior Researcher at CARMA and the U.S. Public Policy Lead within the Public Security Policy Program. Sterlin contributes SME policy knowledge related to incident response, emerging technology, and US and international governance. Prior to CARMA, Sterlin was a Policy Advisor in the Executive Office of the President, wh
Sterlin Waters is a Senior Researcher at CARMA and the U.S. Public Policy Lead within the Public Security Policy Program. Sterlin contributes SME policy knowledge related to incident response, emerging technology, and US and international governance. Prior to CARMA, Sterlin was a Policy Advisor in the Executive Office of the President, where he focused on cyber and emerging technology policy issues at the National Security Council, Cyber and Emerging Technology Directorate and in the Office of the National Cyber Director. His experience included supporting senior national security officials in developing and managing the implementation of defensive and offensive cyber policy, coordinating high-level diplomatic engagements, and contributing to the drafting of government wide cyber strategy. Prior to his time at the White House, Sterlin was a Research Assistant at the Senate Homeland Security and Governmental Affairs Committee, covering cybersecurity and counterterrorism policy. He has also completed internships in the House of Representatives. Sterlin has a Master’s Degree in Security Policy Studies from The George Washington University and a Bachelor’s Degree in Political Science, with a focus on International Relations from the University of Central Florida.
Giulio Corsi is a Senior Researcher at CARMA, focusing on Offense/Defense Dynamics and balances generated by AI capabilities. With extensive experience in evaluating AI safety and the societal impacts of AI systems, Giulio has previously worked on the development of novel techniques for assessing risks in AI-mediated environments. His res
Giulio Corsi is a Senior Researcher at CARMA, focusing on Offense/Defense Dynamics and balances generated by AI capabilities. With extensive experience in evaluating AI safety and the societal impacts of AI systems, Giulio has previously worked on the development of novel techniques for assessing risks in AI-mediated environments. His research, which applies quantitative methods and machine learning techniques to analyze AI risks, has been published in journals such as EPJ Data Science and Harvard Kennedy School Misinformation Review. Giulio is also interested in the exploration of systemic and cascading AI risks across multiple domains, with a focus on interdisciplinary measurement and mapping approaches. Giulio's research has also informed high-impact policy context, and he has recently been contributing to a large-scale assessment, led by the United Nations International Atomic Energy Agency, of how frontier AI can disrupt emergency responses during nuclear emergencies.
Alongside his work at CARMA, Giulio holds a position as a Research Associate at the University of Cambridge's Leverhulme Centre for the Future of Intelligence, where he leads research work on epistemic security, exploring how AI affects information circulation and public decision-making. Giulio holds an MPhil and PhD from the University of Cambridge.
Daniele (Dan) Palombi is a Senior Researcher at CARMA within the Geostrategic Dynamics program. His work at CARMA focuses on combining his strong interest in AI safety research with his expertise in mechanism design and multi-agent systems, designing and implementing mechanisms and markets that promote positive-sum strategic interactions
Daniele (Dan) Palombi is a Senior Researcher at CARMA within the Geostrategic Dynamics program. His work at CARMA focuses on combining his strong interest in AI safety research with his expertise in mechanism design and multi-agent systems, designing and implementing mechanisms and markets that promote positive-sum strategic interactions among intelligent agents, favouring alignment with human values and societal goals, and building mathematical models and simulations for complex strategic scenarios. Alongside his work at CARMA, Dan is a consultant (Mechanism Design, Market Design and Operations Research) at 20squares and coordinates its R&D efforts as a member of its core team, and a researcher at the Institute for Categorical Cybernetics (Programming Language Theory, Type Theory, applications of Category Theory to Game Theory and Multi-Agent Systems). In the past, Dan has worked as a researcher in Concurrent Programming and Probabilistic Programming, as a developer and designer of programming languages, build systems and developer tools, and as an embedded developer for industry-scale manufacturing robots.
Kyle A. Kilian is a part-time member of the Societal Defense Team at CARMA, which includes Public Security Policy, Offense/Defense Dynamics, and crosscutting concerns. He is an accomplished leader in intelligence analysis, multidisciplinary research, and technology modernization in the national security enterprise. He was recently the Dep
Kyle A. Kilian is a part-time member of the Societal Defense Team at CARMA, which includes Public Security Policy, Offense/Defense Dynamics, and crosscutting concerns. He is an accomplished leader in intelligence analysis, multidisciplinary research, and technology modernization in the national security enterprise. He was recently the Deputy Director of the Transformative Futures Institute, where he focused on applying strategic foresight to anticipate risks from emerging technologies. Kyle has served for over a decade in the Defense and Intelligence Community (IC) in strategic, tactical, and joint operational environments. His research interests lie at the intersection of artificial intelligence (AI), complex systems, network modeling, and international security, with expertise in exploratory futures modeling and activity-based intelligence (ABI). Kyle is a senior research fellow at the Center for the Future Mind, a Mentor at the Foresight Institute, and a 2022 fellow with the Global Catastrophic Risk Institute. Kyle holds graduate degrees in Data Science and Cyber Intelligence from the National Intelligence University and International Affairs from the American University's School of International Service.
Joe Rogero is a Senior Risk Assessment Associate at CARMA, where he provides part-time support for the development of CARMA's risk assessment frameworks. Joe is a former Reliability Engineer with a background in risk assessment, root cause analysis, incident investigation, technical writing, and data analysis. His prior engineering work i
Joe Rogero is a Senior Risk Assessment Associate at CARMA, where he provides part-time support for the development of CARMA's risk assessment frameworks. Joe is a former Reliability Engineer with a background in risk assessment, root cause analysis, incident investigation, technical writing, and data analysis. His prior engineering work included the use of failure modes and effect analysis, event tree risk assessment, multidisciplinary facilitation, and other tools and techniques to identify, quantify, aggregate, prioritize, and communicate safety and financial risk scenarios from the mundane to the catastrophic. He has taught the AI Safety Fundamentals course, guiding more than 50 participants through an introduction to AI Safety with BlueDot Impact. He has also volunteered as a facilitator and career navigator for AI Safety Quest. Last year, he began writing for the Communications team at the Machine Intelligence Research Institute.
Avyay Casheekar is a Research Assistant at CARMA, where his work spans a wide range of projects across risk management and policy strategy. He has a growing interest in AI systems and their interactions with society, and has been a cohort mentor for the Axiom Futures Fellowship and the Center for AI Safety's "AI Safety, Ethics, and Societ
Avyay Casheekar is a Research Assistant at CARMA, where his work spans a wide range of projects across risk management and policy strategy. He has a growing interest in AI systems and their interactions with society, and has been a cohort mentor for the Axiom Futures Fellowship and the Center for AI Safety's "AI Safety, Ethics, and Society" course. His experience includes implementing AI solutions for companies and conducting research at Columbia University's Software Systems lab. Previously a fellow at the AI Futures Fellowship, he worked on methods to improve robustness in AI systems to adversarial attacks. He has since been involved in independent research forecasting AI chip technologies and studying reliability patterns in AI systems. Avyay completed the AI Safety Fundamentals course in governance and has acquired practical knowledge of the EU AI Act through his involvement in understanding and analyzing standards from the Joint Technical Committee 21. His focus areas include scalable oversight methods, evaluation methodologies for AI models, and understanding the implications of rapid compute progress. His work aims to contribute to the development of safe and aligned AI systems, focusing on maximizing the positive societal impact of AI. He is keen to expand his contributions to AI Policy by pursuing a long-term career working in helping design Governance Systems and Policies for safe AI.
Pam Massey is a part-time executive assistant at CARMA, bringing over 20 years of experience as a seasoned professional specializing in providing high-level support to C-suite executives. Her career spans human resources, advertising, and contract negotiation, highlighting her versatility and expertise in managing complex administrative f
Pam Massey is a part-time executive assistant at CARMA, bringing over 20 years of experience as a seasoned professional specializing in providing high-level support to C-suite executives. Her career spans human resources, advertising, and contract negotiation, highlighting her versatility and expertise in managing complex administrative functions and strategic initiatives. Pam has a proven track record of managing logistics, handling sensitive information, coordinating intricate schedules, and facilitating seamless communication between executives and stakeholders. Her background in human resources has equipped her with a deep understanding of employee relations and talent management, while her experience in advertising has honed her skills in project coordination and marketing strategy. Known for her professionalism, discretion, and proactive approach, Pam consistently delivers high-quality support and contributes to the success of the teams and leaders she serves.
Anthony Aguirre is the Executive Director of the Future of Life Institute, an NGO examining the implications of transformative technologies, particularly AI. He has published research on AI risk progressions and mitigations, loyal AI, and AI governance. As a professor at UC Santa Cruz, his research has spanned foundational physics to AI p
Anthony Aguirre is the Executive Director of the Future of Life Institute, an NGO examining the implications of transformative technologies, particularly AI. He has published research on AI risk progressions and mitigations, loyal AI, and AI governance. As a professor at UC Santa Cruz, his research has spanned foundational physics to AI policy. Aguirre co-founded Metaculus, a platform leveraging collective intelligence to forecast science and technology developments, and the Foundational Questions Institute, supporting fundamental physics research. With a PhD from Harvard and postdoctoral work at the Institute for Advanced Study, Aguirre's multidisciplinary background enables him to play a significant role in shaping the future of transformative technologies and their societal impact.
Eric Drexler, a pioneering researcher in nanotechnology and AI, is known for his seminal works “Engines of Creation” and “Nanosystems,” which laid the foundation for the field of molecular systems engineering. In recent years, he has focused on the potential development and implications of advanced AI systems. His “Comprehensive AI Servic
Eric Drexler, a pioneering researcher in nanotechnology and AI, is known for his seminal works “Engines of Creation” and “Nanosystems,” which laid the foundation for the field of molecular systems engineering. In recent years, he has focused on the potential development and implications of advanced AI systems. His “Comprehensive AI Services” model explores the emergence of general intelligence through diverse AI systems that perform distinct, interpretable tasks within role architectures. This perspective informs his analysis of the opportunities and challenges posed by AI, as well as strategies for managing risks and harnessing benefits.
Dr. Drexler's current work explores how advances in AI will expand general implementation capacity—the ability of humans to achieve broad goals by designing, developing, deploying, applying, and adapting complex sociotechnical systems at scale. Through his research and writing, he seeks to deepen understanding of the transformative potential of advanced AI capabilities and their implications for potential cooperative global strategies.
Shezaad J. Dastoor is the Program Manager for the Digital Transformation of United Nations Peacekeeping, where he leverages data, technology, and innovation to transform the UN's approach to peace and security. As a seasoned international civil servant working at the confluence of technology and geopolitics, he brings a unique set of skil
Shezaad J. Dastoor is the Program Manager for the Digital Transformation of United Nations Peacekeeping, where he leverages data, technology, and innovation to transform the UN's approach to peace and security. As a seasoned international civil servant working at the confluence of technology and geopolitics, he brings a unique set of skills and perspectives to CARMA's advisory board.
He has a proven track record of drafting analytical products, statements, and policy communications, incorporating elements of game theory and risk management to facilitate decision-making processes for UN leadership, including the Secretary-General. His prior roles include serving as Special Assistant and Advisor to the Chief of Staff for UN Peacekeeping, where he coordinated strategic initiatives, crisis management, and stakeholder engagement. He also served as an intelligence analyst in South Sudan and a liaison officer in Afghanistan, providing actionable insights and risk mitigation strategies in high-stakes environments. Before his tenure at the UN, Shezaad worked at the World Bank, where he focused on strategic planning and international development projects. He is also a co-founder of LINC Negotiation Architects, a firm specializing in negotiation analysis, training, and simulations. Additionally, he serves on OpenAI's Red Team Network, assessing risks and policy implications of AI models and systems.
Shezaad holds a master’s degree in international peace and conflict resolution from American University and a bachelor’s degree in political science from St. Xavier’s College, Mumbai. Fluent in English, Hindi, and Urdu, he is dedicated to advancing global stability through innovative approaches to AI risk management and strategic operations.
Center for AI Risk Management & Alignment
Copyright © 2025 Center for AI Risk Management & Alignment - All Rights Reserved. The Center for AI Risk Management & Alignment is a project of Social & Environmental Entrepreneurs, Inc., a 501(c)(3) nonprofit public charity.
We use cookies to analyze website traffic and optimize your website experience. By accepting our use of cookies, your data will be aggregated with all other user data.