Qualifications
Qualifications• Advanced degree (Ph.D. preferred) in AI, computer science, social science, or a related field.• Proven experience in research within AI dynamics, risk assessment, or a related discipline.• Strong analytical and quantitative skills, with proficiency in modeling and simulation techniques.• Excellent communication skills, both written and verbal, to convey complex concepts to a broad audience.• Ability to work independently and collaboratively in a remote setting.
About the job
Job Overview
As artificial intelligence technologies continue to evolve at a rapid pace, it is crucial to bridge the knowledge gap surrounding the intricate dynamics that dictate whether AI systems serve as threats or safeguards to society. In this pivotal role, you will spearhead research initiatives aimed at elucidating these offense-defense dynamics. You will investigate how particular characteristics of AI technologies affect their likelihood to either bolster societal safety or exacerbate risks. Utilizing interdisciplinary approaches, you will construct both quantitative and qualitative frameworks to evaluate the dissemination of AI capabilities within society, whether as protective or harmful applications. Your insights will be instrumental for developers, evaluators, standards organizations, and policymakers in anticipating and mitigating potential risks. This position is fully remote but may require occasional travel.
About CARMA
The Center for AI Risk Management & Alignment (CARMA) is dedicated to assisting society in navigating the complex and potentially catastrophic risks associated with advanced AI systems. Our mission is to minimize the dangers posed to humanity and the biosphere by transformative AI technologies.
We focus on grounding AI risk management in rigorous analytical frameworks, developing policy structures that directly address artificial general intelligence (AGI), advancing technical safety measures, and promoting global perspectives on long-term safety. Through these interconnected strategies, CARMA aims to provide essential support to society in managing the significant risks posed by advanced AI before they become reality.
CARMA operates as a fiscally-sponsored project under Social & Environmental Entrepreneurs, Inc., a registered 501(c)(3) nonprofit organization.
Key Responsibilities
• Create quantitative models that capture the interplay between technological, social, and institutional factors impacting AI risk landscapes.
• Develop comprehensive analytical models and simulations to pinpoint critical leverage points for policy interventions that could alter offense-defense dynamics towards safer outcomes.
• Enhance and operationalize our existing offense-defense dynamics taxonomy and foundational framework, crafting metrics and models to predict whether specific AI system traits favor offensive or defensive uses.
• Conduct empirically-informed analyses and synthesize findings into actionable recommendations.