companyArena Intelligence logo

Data Scientist at Arena Intelligence | Bay Area

On-site Full-time

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Unlock Your Potential

Generate Job-Optimized Resume

One Click And Our AI Optimizes Your Resume to Match The Job Description.

Is Your Resume Optimized For This Role?

Find Out If You're Highlighting The Right Skills And Fix What's Missing

Experience Level

Senior

Qualifications

Your ResponsibilitiesAnalyze extensive, complex datasets to discover trends, biases, and causal relations in model behavior and system performance. Formulate and test hypotheses regarding data quality, evaluation outcomes, and model performance through experimental design. Create reproducible analysis pipelines utilizing Python, Pandas, NumPy, and Spark for processing large-scale data. Collaborate with ML researchers and engineers to develop metrics and analyses assessing model performance across various domains, prompts, and tasks. Establish causal reasoning frameworks and statistical methodologies to elucidate model behaviors and performance. Effectively communicate findings through various channels, including blog posts and presentations.

About the job

Join Arena Intelligence as a Data Scientist

Arena Intelligence stands at the forefront of AI evaluation, offering an open platform that examines how AI models perform in real-world scenarios. Founded by UC Berkeley's SkyLab researchers, our mission is to push the boundaries of AI utility.

Each month, millions engage with Arena Intelligence to assess the performance of pioneering AI systems, using our community's insights to foster transparent, comprehensive, and human-centric model evaluations. Leading enterprises and AI labs depend on our evaluations to gauge real-world reliability, alignment, and impact. Our leaderboards are regarded as the benchmark for AI performance, trusted by industry leaders and influencing global discussions on model reliability and advancement.

Our diverse team of researchers, engineers, and builders hail from prestigious institutions such as UC Berkeley, Google, Stanford, DeepMind, and Discord. We prioritize truth, agility, and craftsmanship while fostering an environment that values curiosity and impact over hierarchy. At Arena, skilled individuals from all backgrounds are empowered to excel in their fields, contributing to an atmosphere rich in excellence, energy, and focus.

The Role

As a Data Scientist, you will investigate and interpret the data that fuels millions of AI evaluations weekly. Your responsibilities will include generating and testing hypotheses, identifying causal relationships, and revealing insights that enhance our understanding of frontier model behaviors in practical applications. You will collaborate with machine learning researchers and engineers to design experiments, analyze extensive datasets, and develop statistical frameworks aimed at refining the reliability and interpretability of our AI evaluation systems. Senior-level candidates are preferred for this role.

About Arena Intelligence

Arena Intelligence is an innovative platform revolutionizing AI model evaluation, driven by a commitment to transparency and community engagement. With insights from our extensive user base, we continuously enhance our evaluation methodologies, ensuring they meet the evolving challenges of the AI landscape.

Similar jobs

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.