Principal Research Scientist - AI Safety

Why join Faculty?Founded in 2014, Faculty is on a mission to harness the transformative power of AI, believing it to be the most pivotal technology of our era. With a diverse portfolio of over 350 global clients, we have consistently driven performance improvements through human-centric AI solutions. Discover our tangible impacts here.We focus on responsible AI development rather than chasing trends. Our team excels in innovating, building, and deploying AI solutions that truly matter. We offer unmatched expertise in technology, product development, and service delivery across various sectors including government, finance, retail, energy, life sciences, and defense.As our reputation grows, so does our commitment to finding individuals who share our passion for intellectual curiosity and the ambition to create a positive legacy through technology.AI defines this epoch; join us in exploring its most impactful applications and turning them into reality.About Our TeamOur dedicated team at Faculty engages in vital red teaming and creates evaluations for misuse capabilities in critical domains such as CBRN, cybersecurity, and international security. Our contributions have been recognized, notably in OpenAI's system card for o1.We are committed to conducting foundational research on mitigation strategies, sharing our findings through peer-reviewed conferences, and with national security institutions. Furthermore, we design evaluations for model developers focusing on safety and societal impacts of advanced AI models, showcasing our extensive expertise in the safety domain.About The RoleAs the Principal Research Scientist for AI Safety, you will lead Faculty's dynamic research team, influencing the future of safe AI systems. Your role will encompass overseeing the scientific research agenda centered on large language models and other significant systems. You will guide fellow researchers, spearhead external publications, and align your efforts with Faculty’s mission to develop trustworthy AI, allowing you to make a substantial impact in this fast-evolving field.

Why Join Faculty?Founded in 2014, Faculty is at the forefront of artificial intelligence, believing it to be the most transformative technology of our era. Over the years, we have partnered with more than 350 clients globally, enhancing their performance through human-centered AI solutions. Our tangible impact can be explored here.We prioritize genuine innovation over fleeting trends. Our commitment lies in developing and implementing responsible AI that significantly influences outcomes. Our diverse clientele includes sectors such as government, finance, retail, energy, life sciences, and defense, all of whom benefit from our profound expertise in technology, product development, and delivery.Our rapidly expanding business and reputation drive us to seek individuals who share our passion for intellectual exploration and aspire to create a positive technological legacy.AI is a groundbreaking technology; at Faculty, you will have the freedom to conceive its most impactful applications and bring them to fruition.About Our TeamThe Research team at Faculty is dedicated to critical red teaming and developing evaluations for misuse capabilities in sensitive domains, including CBRN, cybersecurity, and international security. We collaborate with leading frontier model developers and national safety institutes, and our contributions have been recognized in OpenAI's system card for o1.We also engage in fundamental technical research focused on mitigation strategies, with our findings presented at peer-reviewed conferences and shared with national security organizations. Additionally, we create evaluations for model developers across various safety-related areas, highlighting our comprehensive expertise in the safety domain.Role OverviewWe are in search of a Senior Research Scientist to join our high-impact R&D team. You will spearhead innovative research that enhances scientific understanding and drives our goal of developing safe AI systems. This is a vital role within a small, empowered team that conducts essential red teaming and evaluations for frontier models in sensitive fields such as cybersecurity and national security, allowing you to influence the future of safe AI deployment in real-world scenarios.

physicsx seeks a Principal Research Scientist based in London, United Kingdom. This position centers on guiding research projects that introduce new concepts and methods in physics. The work plays a key part in supporting the company’s mission to drive technological advancement through scientific discovery. Role overview As Principal Research Scientist, the main responsibility is to lead innovative research efforts. Projects will focus on developing and testing original approaches within the field of physics. The results of this research are intended to contribute directly to the company’s ongoing progress in technology. What you will do Lead and manage research projects from concept to completion Develop new ideas and experimental approaches in physics Ensure research aligns with the company’s goals for scientific and technological progress Location This role is based in London, United Kingdom.

Application Deadline: We are actively interviewing candidates and aim to fill this position promptly with the right individual.ABOUT THE ROLEAt Apollo Research, we conduct evaluations that meticulously assess the risks associated with advanced AI systems. Collaborating with leading laboratories such as OpenAI, Anthropic, and Google DeepMind, you will have the unique opportunity to engage with cutting-edge AI models ahead of their public release. The ideal candidate possesses a passion for rigorously testing state-of-the-art AI technologies and excels in creating and automating efficient evaluation pipelines.YOUR RESPONSIBILITIES WILL INCLUDE- Conducting pre-deployment evaluation campaigns on the world's most advanced AI systems. Our partnerships with various labs provide access to a wide range of models that no single lab can offer, allowing you to be among the first to engage with new models.- Exploring AI cognition by analyzing extensive model transcripts to identify novel behavioral patterns that have yet to be documented. These insights can be both surprising and enlightening, including phenomena like non-standard language and reward-seeking reasoning as discussed in our anti-scheming paper.- Developing new evaluations for frontier risks, creating innovative test environments, and scaling these across multiple scenarios.- Collaborating with leading AI developers to share your insights, receive feedback, and ensure your evaluations influence the deployment strategies of the most advanced AI systems.- Optimizing and automating the evaluation pipeline. We utilize automation in building, executing, and analyzing evaluations. As agent capabilities evolve rapidly, you will have the autonomy to reshape the pipeline to keep pace with these advancements.KEY REQUIREMENTS

Application Deadline: We are actively interviewing candidates and seek to fill this position promptly with a suitable applicant. THE OPPORTUNITYBecome a vital member of our groundbreaking AGI safety product team and play a key role in transforming intricate AI research into actionable tools aimed at minimizing AI-related risks. In your role as an applied researcher, you will collaborate closely with our CEO (who also acts as Head of Product), product engineers, and the Evals team’s software engineers to develop solutions that enhance AI agent safety for our clients. Currently, we are concentrating on the oversight of AI coding agents to identify failures in safety and security. You will be part of a compact team, which allows you to significantly influence both team dynamics and technological approaches while quickly assuming greater responsibilities. This position is perfect for you if you have a fervent desire to employ empirical research methodologies to enhance the safety of AI systems in practical applications. If you relish the challenge of converting theoretical AI risks into tangible detection mechanisms, thrive in fast-paced environments, and are eager to see your research make a meaningful impact on real-world AI safety, then we would love to hear from you.KEY RESPONSIBILITIESResearch & Development- Collect and catalog coding agent failure modes systematically from real-world instances, public examples, research literature, and theoretical predictions.- Design and execute experiments to evaluate monitor effectiveness across various failure modes and agent behaviors.- Develop and maintain evaluation frameworks to track advancements in monitoring capabilities.- Refine monitoring strategies based on empirical findings, optimizing detection accuracy alongside computational efficiency.- Stay updated with the latest research in AI safety, agent failures, and detection methodologies.- Keep abreast of advancements in coding security and safety vulnerabilities.Monitor Design & Optimization- Create a comprehensive library of monitoring prompts tailored to specific failure modes (e.g., security vulnerabilities, goal misalignment, deceptive behaviors).- Experiment with various reasoning strategies and output formats to enhance monitor reliability.- Design and evaluate hierarchical monitoring architectures and ensemble approaches.

Become a Catalyst for Change at Axon.At Axon, we are committed to safeguarding lives and enhancing public safety through innovative technology. As pioneers in the field, we tackle critical issues surrounding safety and justice with our comprehensive ecosystem of devices and cloud-based software solutions. Our collaborative culture thrives on open communication and diverse perspectives, both from our customers and within our teams.Working at Axon is not just a job; it's a chance to make a tangible difference. Here, you will take initiative and make a real impact while growing personally and professionally in a fast-paced, meaningful environment.Your RoleWe are looking for adept and forward-thinking Machine Learning Scientists to enhance our AI team, focusing on applications of AI, particularly in Large Language Models (LLM) and Computer Vision across Cloud, Devices, and Robotics. As an integral part of our research and development initiatives, you will significantly contribute to advancing cutting-edge technologies in LLMs, Multimodal Large Language Models (MLLMs), Computer Vision, and Generative AI for law enforcement applications and more. You will work collaboratively with multidisciplinary teams to innovate, develop, and implement advanced LLM, MLLM, and CV models, enabling intelligent reasoning and perception of multimodal data.

Join InstaDeep as a Research Scientist where you will drive innovation in decision-making AI. Collaborate with leading experts to develop cutting-edge machine-learning models, conduct impactful experiments, and contribute to groundbreaking research in computational biology. Your insights will help expand our scientific knowledge and support our partnerships with global leaders like BioNTech SE and Google DeepMind.

aisi seeks a Research Engineer or Research Scientist to focus on Model Transparency in London, UK. The position involves research aimed at making AI models more understandable and accountable. The goal is to clarify complex behaviors within these systems and help build trust in their outcomes. Key responsibilities Research new ways to improve the interpretability of AI models Create methods and tools that help explain how models make decisions Tackle challenges in understanding and communicating model behavior Contribute to initiatives that support greater accountability in AI Location This role is based in London, UK.

HeartFlow Inc. is at the forefront of medical technology, revolutionizing the diagnosis and management of coronary artery disease—the leading cause of death globally. Utilizing advanced technology, our flagship product, the HeartFlow FFRCT Analysis, is an AI-driven, non-invasive cardiac test endorsed by the ACC/AHA Chest Pain Guidelines. This innovative solution generates a color-coded, 3D model of a patient's coronary arteries, highlighting the impact of blockages on blood flow to the heart. HeartFlow stands out as the first AI-driven non-invasive integrated heart care solution throughout the CCTA pathway, enabling clinicians to identify stenoses (RoadMap™ Analysis), assess coronary blood flow (FFRCT Analysis), and characterize and quantify coronary atherosclerosis (Plaque Analysis). Our expanding pipeline of products and team are dedicated to transforming the landscape of precision heart care.As a publicly traded company (HTFL), HeartFlow has garnered international acclaim for its groundbreaking contributions to healthcare innovation. We are supported by medical societies worldwide and have received clearances for use in the US, UK, Europe, Japan, and Canada, serving over 500,000 patients globally.We invite you to join HeartFlow's distinguished Research team as a Collaborating Research Scientist, with a joint visiting researcher appointment in Professor Ben Glocker’s lab at Imperial College London. This role centers on the advancement of state-of-the-art Generative AI technologies for Coronary Computed Tomography Angiography (CCTA). You will engage in pioneering academic research, produce intellectual property, and contribute to high-impact publications, ensuring that technical innovations are effectively translated into product-grade models at HeartFlow.At Imperial College, you will become an integral member of one of the world’s premier medical imaging laboratories, participating in seminars, collaborating with leading academics, and immersing yourself in the dynamic ecosystem of South Kensington.At HeartFlow, you will play a crucial role in shaping our research roadmap, gaining access to extensive proprietary datasets not available in purely academic settings, and translating your findings into medical devices with a global impact.Key Responsibilities:Design and implement cutting-edge generative architectures to synthesize high-fidelity, anatomically accurate CCTA image data.Publish and present findings at prestigious conferences (e.g., MICCAI, ICCV, CVPR, NeurIPS).Provide technical reports and reproducible code to facilitate the transition of research into products by Research team members and development teams.Identify and protect new intellectual property for HeartFlow in emerging AI domains.

Join Anthropic as an AI Safety Fellow, where our mission is to develop reliable and interpretable AI systems that prioritize safety and societal benefits. This unique opportunity allows you to engage in cutting-edge AI safety research for four months, with guidance from leading experts in the field. You will utilize external infrastructures to undertake an empirical project aligned with our research goals, culminating in a public output such as a research paper. We foster a collaborative environment, connecting you with the broader AI safety research community, and providing generous stipends and resources to support your research endeavors.

At Google DeepMind, we celebrate the rich diversity of experiences, knowledge, backgrounds, and perspectives that contribute to our mission of creating extraordinary impact. We are dedicated to providing equal employment opportunities regardless of sex, race, religion or belief, ethnic or national origin, disability, age, citizenship, marital status, sexual orientation, gender identity, pregnancy, or any other legally protected status. If you require accommodation due to a disability or additional need, please let us know.About UsArtificial Intelligence stands as one of humanity’s most transformative inventions. At Google DeepMind, our team of scientists, engineers, and machine learning experts work collaboratively to push the boundaries of AI technology. We harness our innovations for the greater good and scientific progress, prioritizing safety and ethics in every endeavor.SnapshotWe are in search of talented engineers and scientists to help propel our research in Artificial Intelligence as part of our mission to develop the next generation of Generative AI technologies. Our team is uniquely positioned to work on speculative, long-term projects that, when successful, yield significant advancements in the capabilities of our frontier models. We are committed to exploring paradigm-shifting concepts and investing the necessary time and resources to realize them. Our aim is to ideate, research, develop, validate, and incorporate these advancements into major Google technologies.Our primary research focus is on Gemini Diffusion, our cutting-edge text diffusion model, which we believe has the potential to unlock a new era in generative AI, delivering unmatched capabilities with rapid latency. Backed by GDM leadership, our team is devoted to thoroughly investigating and advancing this groundbreaking technology.We are a multi-disciplinary group of engineers and scientists who collaborate closely, leveraging our diverse skills with a strong emphasis on developing and deploying innovative AI solutions. Our problem-focused ethos drives our interest in any tool or idea that can effectively address real-world challenges.

At Google DeepMind, we prioritize diversity in experience, knowledge, backgrounds, and perspectives, leveraging these attributes to create remarkable impact. We are dedicated to providing equal employment opportunities regardless of sex, race, religion, belief, ethnic or national origin, disability, age, citizenship, marital status, domestic or civil partnership status, sexual orientation, gender identity, pregnancy, or any other condition protected by applicable law. If you have a disability or additional needs that require accommodation, please feel free to reach out to us.SnapshotWe seek talented Research Scientists to advance fundamental research and technology in Artificial Intelligence, as part of our interdisciplinary and collaborative Reinforcement Learning team.About UsDeepMind’s RL team is a cohesive and collaborative group of scientists and engineers, led by Tom Schaul. We address large-scale research challenges in reinforcement learning, designing, refining, and scaling RL algorithms to yield significant scientific and product impact. Over the past decade, our RL team has been pivotal in developing innovations such as DQN, AlphaGo, Rainbow, AlphaZero, MuZero, AlphaStar, AlphaProof, and Gemini. Join us in creating the next groundbreaking advancement!The RoleAs a Research Scientist, you will leverage your machine learning expertise and technical skills to innovate, spearhead research projects, and apply findings to impactful challenges. You will be responsible for implementing code, conducting experiments, owning results from start to finish, communicating findings effectively, and collaborating with fellow team members to empower others.Your work may involve:Initiating and pursuing novel research directions by proposing and testing hypotheses.Implementing algorithmic ideas and executing comprehensive experiments, including setup, execution, analysis, and iteration.Sharing your expertise and insights with other researchers.Building or enhancing infrastructure to support scalable research.Designing evaluations and ablations that address real questions and influence perspectives.Meticulously analyzing results, including debugging and failure analysis.Effectively communicating through visualizations, write-ups, and publication-ready narratives and figures.Contributing to a culture of first-principles thinking, high standards, and constructive feedback.Our projects encompass a broad spectrum of state-of-the-art machine learning and AI domains, including large language models, distributed machine learning, and more.

Application Deadline: We are actively conducting interviews and aim to onboard the right candidate promptly.ABOUT THE ROLEAt Apollo Research, we are embarking on an ambitious journey to establish a groundbreaking field known as the “Science of Scheming.” We are seeking passionate Research Scientists and Engineers eager to contribute to the creation of a new hard science from the ground up.YOUR RESPONSIBILITIES- Collaborate with leading AI innovators. Our partnerships with various labs provide access to an extensive range of models that no single AI lab can offer. Your contributions will significantly influence the development and deployment of advanced AI systems.- Conduct in-depth studies of reinforcement learning dynamics that drive the emergence of reward-seeking behaviors, evaluation awareness, and misaligned preferences. Design and train model organisms, scaling insights to cutting-edge systems.- Explore the “Scaling Laws of Scheming.” Establish empirical foundations to forecast how scheming risks evolve as models increase in capability.- Innovate evaluation techniques that can potentially scale to highly evaluation-aware models.- Investigate AI cognition. Uncover unique patterns in the reasoning processes of advanced AI systems that have yet to be documented.Note: This position does not focus on interpretability roles.KEY REQUIREMENTSWe seek a diverse set of skills to advance our research agenda. We do not expect any single candidate to meet all the criteria listed below. However, a successful candidate will likely excel in one or several of the following areas:- Fast-paced empirical research: Ability to design and execute experiments with a focus on accelerating iteration cycles.

Who are we?At Cohere, we are driven by a powerful mission: to scale intelligence for the betterment of humanity. Our focus is on training and implementing state-of-the-art models that empower developers and enterprises in creating AI systems that foster innovative experiences, including content generation, semantic search, retrieval-augmented generation (RAG), and intelligent agents. We believe our contributions are pivotal in accelerating the global adoption of AI technologies.Our passion is reflected in the meticulous work we do. Each team member plays a crucial role in enhancing our models' capabilities and the value they provide to our clients. We embrace a culture of hard work and agility, always prioritizing the needs of our customers.Cohere comprises a diverse group of researchers, engineers, designers, and professionals who are all experts in their respective fields. We understand that a variety of perspectives is essential for creating exceptional products.Join us in our quest to shape the future of AI!Why join Cohere Labs?Cohere Labs functions as the dedicated research division of Cohere. Our goal is to push the frontiers of machine learning by addressing complex challenges and unlocking new avenues for exploration. We believe that the future of AI will be determined not only by technical innovations but also by fostering environments where diverse communities of researchers, engineers, and interdisciplinary experts can collaborate on bold ideas.We pursue our mission through two main strategies:Establishing open research environments that facilitate collaboration across various backgrounds in cutting-edge ML and NLP research.Enhancing access to research training, welcoming new voices and perspectives from around the globe.About the roleAs a Senior Research Scientist at Cohere Labs, you will spearhead an independent and impactful research agenda focused on advanced topics in AI. You will have the autonomy to pose ambitious questions, delve into unexplored areas, and share your findings with the broader community. We are particularly interested in research that not only advances scientific knowledge but also addresses the societal implications of machine learning.This position is ideal for someone who excels at both leading initiatives and fostering learning: driving cutting-edge research through global collaborations, mentoring emerging researchers, and broadening participation in ML research.You may be a fit if you:Hold a PhD or possess equivalent research experience...

Join the prestigious Anthropic Fellows Program, focused on advancing AI Safety. As a Fellow, you will collaborate with distinguished researchers and contribute to groundbreaking projects that prioritize the ethical development and deployment of artificial intelligence.

Isomorphic Labs is pioneering the application of advanced AI technologies to unlock profound scientific insights, expedite breakthroughs, and develop transformative medicines with a vision to eradicate all diseases.The future is on the horizon, made possible by the remarkable capabilities of machine learning. We envision a world where diseases are effectively managed or eliminated through accelerated and enhanced drug discovery.Join our interdisciplinary team to spearhead groundbreaking innovations and play a pivotal role in our mission to achieve ambitious health-related goals, all while contributing to a vibrant and collaborative workplace culture.The future we aspire to is being shaped today. It begins with our culture at Isomorphic Labs. It starts with you.About Isomorphic LabsFounded in 2021, Isomorphic Labs (IsoLabs) strives to enhance human health by building upon the Nobel Prize-winning AlphaFold system. Our diverse team of experts in drug discovery and machine learning has developed groundbreaking predictive and generative AI models that accelerate scientific discovery at an unprecedented pace.Our name reflects our belief in the inherent symmetry between biology and information science. By leveraging AI's robust capabilities, we can model complex biological processes to design innovative molecules, predict drug performance, and develop novel treatments for some of the world's most challenging diseases.Our state-of-the-art drug design engine integrates AI models capable of addressing various therapeutic areas and drug modalities. We are dedicated to continuously innovating model architectures and creating cutting-edge solutions to advance rational drug design.With each significant advancement, we are progressing towards the promise of digital biology, moving closer to our ambitious mission of ultimately solving all diseases through the power of AI.

About AnthropicAt Anthropic, we are dedicated to the advancement of safe, interpretable, and steerable AI systems. Our mission is to ensure that AI technologies remain beneficial to users and society as a whole. Our rapidly expanding team consists of passionate researchers, engineers, policy experts, and business leaders collaborating to create cutting-edge AI solutions.About the Role:As a Research Engineer focused on Alignment Science, you will design and execute sophisticated machine learning experiments aimed at understanding and guiding the behavior of powerful AI systems. You are driven by a desire to make AI systems helpful, honest, and harmless, and you recognize the complexities associated with human-level capabilities. This role requires a blend of scientific inquiry and engineering expertise. You will engage in exploratory research concerning AI safety, addressing potential risks associated with advanced future systems (classified as ASL-3 or ASL-4 according to our Responsible Scaling Policy), frequently collaborating with teams focused on Interpretability, Fine-Tuning, and the Frontier Red Team.For insights into our ongoing research, visit our blog. We are currently looking to expand our London team in the following research areas:AI Control: Developing methodologies to ensure that advanced AI systems remain safe and non-threatening in unpredictable or adversarial environments.Alignment Stress-testing: Implementing innovative alignment stress-testing frameworks to evaluate AI system resilience.

At Spotify, we're revolutionizing the way our 600 million listeners across 180 global markets engage with music, podcasts, and audiobooks. Our Personalization team plays a pivotal role in delivering tailored experiences that not only meet user expectations but also encourage exploration of new content. We are on the lookout for a talented Research Scientist specializing in machine learning to help us elevate our personalization strategies to new heights.As a key member of our dynamic research team, you will engage in groundbreaking research, development, and integration of innovative generative AI technologies into our search and recommendation systems. Your work will contribute to creating interactive experiences that redefine user engagement with our platform.Join us in an interdisciplinary environment focused on maintaining and advancing the technological frontiers of Spotify. By publishing your findings, you will not only enhance our internal practices but also contribute to the broader research community. Our team enjoys strong collaborations with both internal product groups and external research entities.Key Expertise Areas: Information Retrieval, Recommendation Systems, Language Technologies, Foundational Models, Generative AI Technologies, and Machine Learning.

Why Join Faculty?Founded in 2014, Faculty believes that artificial intelligence (AI) will shape the future of technology. With over 350 global clients, we have consistently delivered transformative AI solutions that engage human-centric approaches. Explore our impact here.We prioritize innovation over trends, focusing on responsible AI that drives significant results. Our team combines unmatched technical, product, and delivery expertise, serving clients across various sectors including government, finance, retail, energy, life sciences, and defense.As our business and reputation flourish, we seek individuals who share our passion for intellectual exploration and a commitment to leaving a lasting technological legacy.Join us in harnessing AI’s potential to create powerful solutions that will define the future.About Our TeamIn our Professional and Financial Services division, we leverage over a decade of applied AI experience to guide our clients through a rapidly evolving landscape.We design and implement AI solutions that enhance operational efficiency, improve customer experiences, and unlock commercial opportunities in uncertain markets. Operating within a highly regulated environment, we embrace the potential for meaningful innovation while setting the benchmark for technical excellence and secure deployment.About the RoleAs a Principal Data Scientist at Faculty, you will be a technical authority and domain expert, responsible for leading the delivery of intricate, high-impact projects for our key clients. You will establish the technical vision for the data science team while mentoring and developing our data scientists to ensure they remain at the forefront of the industry.Your ResponsibilitiesActing as a technical expert on machine learning, generative AI, and other advanced data science methodologies to deliver innovative solutions.Driving the execution of high-stakes projects while ensuring the highest standards of quality and technical excellence.

Role overview physicsx is seeking a Research Scientist based in London, United Kingdom. The position centers on advancing scientific research and contributing to projects that push the boundaries of physics and technology. The team brings together scientists and engineers who collaborate to tackle challenging technical problems. What you will do Use scientific research methods to develop new concepts and solutions Collaborate with colleagues from a range of technical backgrounds Support projects that investigate emerging areas in physics and technology