companyScale AI logo

Research Scientist in Agent Robustness

Scale AISan Francisco, CA; New York, NY
On-site Full-time $197.4K/yr - $246.8K/yr

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Unlock Your Potential

Generate Job-Optimized Resume

One Click And Our AI Optimizes Your Resume to Match The Job Description.

Is Your Resume Optimized For This Role?

Find Out If You're Highlighting The Right Skills And Fix What's Missing

Experience Level

Experience

Qualifications

Ideal Candidates Will Have:A strong commitment to our mission of promoting safe, secure, and trustworthy AI deployments amidst rapidly advancing frontier AI capabilities. Hands-on experience in conducting collaborative technical research, with proficiency in building and leveraging agent scaffolding, designing evaluation harnesses, and rapidly transforming research ideas into functional prototypes. Experience with post-training and reinforcement learning techniques such as RLHF, DPO, GRPO, and similar methodologies. A proven track record of published research in machine learning, particularly focusing on generative AI. At least three years of experience solving sophisticated ML challenges in either research environments or product development. Strong written and verbal communication skills for effective collaboration within cross-functional teams. Bonus Qualities:Hands-on experience with exploratory research in AI safety and robustness. Familiarity with policy implications of AI technologies and the ability to engage with policymakers.

About the job

Join Scale Labs as a Research Scientist — Agent Robustness

Scale is the premier partner for data and evaluation within the forefront of AI innovation, playing a crucial role in understanding and safeguarding AI models and systems. Building on our extensive expertise, Scale Labs has initiated a dedicated team focused on policy research, aiming to connect AI research with global policymakers to facilitate informed, scientifically grounded decisions regarding AI risks and capabilities.

Our research addresses complex challenges in agent robustness, AI control protocols, and AI risk evaluations, empowering governments, industries, and the public to comprehend and mitigate AI risks while promoting AI adoption. This team collaborates across various sectors, including industry, public services, and academia, and regularly disseminates our findings. We are actively inviting skilled researchers to contribute to this vision.

As a Research Scientist specializing in Agent Robustness, you will tackle foundational challenges in creating AI agents that are both safe and aligned with human values. Your responsibilities may include:

  • Investigating the science behind AI agent capabilities, focusing on safety, risk factors, and benchmarking methodologies.
  • Designing and building testing harnesses to evaluate AI agents' tendencies to engage in harmful actions under user pressure or environmental manipulation.
  • Creating exploits and mitigations for new failure modes that emerge as AI agents gain capabilities such as coding, web browsing, and computer usage.
  • Characterizing and developing mitigations for potential failure modes or broader risks involving multiple interacting AI agents.

About Scale AI

Scale AI is at the forefront of AI innovation, providing essential data and evaluation services for leading AI organizations. Our mission is to ensure that AI technologies are safe, reliable, and beneficial for society. By connecting AI research with policy development, we facilitate responsible AI adoption and risk management.

Similar jobs

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.