Technical Staff Member - Model Evaluations

Our MissionAt Reflection AI, we are dedicated to creating accessible open superintelligence for everyone.Our team is composed of top-tier AI researchers and innovators from prestigious organizations like DeepMind, OpenAI, Google Brain, Meta, Character.AI, Anthropic, and more. We are committed to building open weight models for individuals, enterprises, and even nation states.About the RolePerform essential comparative analyses to deepen our insights into model capabilities.Design and enhance evaluation systems and processes that establish robust feedback loops between data, evaluations, and model behavior.Create generalizable evaluation frameworks that effectively capture reasoning, alignment, and practical usefulness.Collaborate closely with pre-training, post-training, and applied teams to translate insights into tangible model improvements.Expand the boundaries of measurable metrics, utilizing synthetic evaluations, human feedback, and real-world interaction data.About YouProficient in statistical analysis and experimental design, with the ability to rigorously measure model advancements.Knowledgeable in LLM evaluation methodologies, including static benchmarks, human preference evaluations, and agentic tasks.Possess a high degree of agency and thrive in a fast-paced startup atmosphere, prioritizing impact over rigid processes.Eager to work in a pioneering lab, shaping how we measure and accelerate the development of more capable models.Collaborative, detail-oriented, and driven by the desire to create effective feedback loops that enhance model performance.What We Offer:We believe in building superintelligence that is genuinely open, starting from the ground up. Joining Reflection means you will be part of a small, talent-dense team where you will help shape our future and push the boundaries of open foundational models.You will have the opportunity to engage in the most impactful work of your career, knowing that you and your loved ones are well-supported.Competitive Compensation: Salary and equity structured to attract and retain top global talent.Health & Wellness: Comprehensive medical, dental, vision, life, and disability insurance.

Join Us in Revolutionizing AI InfrastructureAt Meter, we are pioneering the application of cutting-edge AI technology to transform the way the internet is constructed, monitored, and managed.Our vertical integration encompasses the entire enterprise networking stack: from hardware and firmware to operating systems and operations. This unique position offers us comprehensive visibility and control over the entire stack via a singular API, along with a proprietary dataset that is unmatched in the industry, paving the way for complete end-to-end automation. Our solutions are already in use by Fortune 500 companies, educational institutions, manufacturing facilities, and cloud-scale clients.We are in the process of assembling a founding core engineering team dedicated to developing and training models that can comprehend these systems, enhance operational efficiency, predict failures, and resolve issues proactively. In essence, you will be instrumental in creating the decision-making framework that underpins the infrastructure of the modern world.You will collaborate closely with our founders, playing a key role in shaping the future of one of the most impactful applications of models available today.Learn more about us at meter.ai.

At Catalog, we are pioneering the commerce infrastructure for AI—creating the essential framework that enables digital agents to not only explore the web but also comprehend, analyze, and engage with products. Our innovations drive the future of AI-driven shopping experiences, fundamentally transforming how consumers discover and purchase items online.Role OverviewAs a Technical Staff Member, you will be instrumental in developing core systems, shaping our engineering culture, and transitioning our vision from prototype to a robust platform. This role requires full-stack expertise and a commitment to owning and resolving challenges from start to finish.Who You AreYou have experience creating beloved and trusted products from the ground up.You combine technical proficiency with a keen product sense and data-driven intuition.You are well-versed in AI technologies.You prioritize speed, write clean code, and ensure thorough instrumentation.You seek a high level of ownership within a small, talent-rich team based in San Francisco.Challenges You Will TackleDevelop and deploy agentic-search APIs that deliver structured and real-time product data in milliseconds.Build checkout systems enabling agents to conduct transactions with any merchant.Create an embeddings and retrieval layer that optimizes recall, precision, and cost efficiency.Establish a product graph and ranking pipeline that adapts based on actual user outcomes.Preferred QualificationsProven experience shipping data-centric products in a live environment.Experience with recommendation systems or information retrieval methodologies.Familiarity with API development, search indexing, and data pipeline construction.Our Work CultureWe operate with a small, high-trust, and highly motivated team, fostering an environment of in-person collaboration in North Beach, San Francisco. Our process involves debate, decision-making, and execution.If your profile aligns with our needs, we will contact you to arrange 2-3 brief technical interviews, followed by an onsite meeting in our office where you will collaborate on a small project, exchange ideas, and meet the team.

Join our dynamic team at Adyen as a Technical Staff Member in San Francisco! We are seeking innovative minds passionate about technology and problem-solving. In this role, you will collaborate with cross-functional teams to craft solutions that enhance our services and improve customer experiences.

About TierZero TierZero helps engineering teams use AI to build and ship code more efficiently. The platform targets the bottleneck of human speed in production, giving teams tools for faster incident response, better operational visibility, and shared knowledge. TierZero is backed by $7M in funding from investors including Accel and SV Angel. Companies like Discord, Drata, and Framer trust TierZero to strengthen their infrastructure for AI-driven engineering. Role Overview: Founding Member of Technical Staff This is an on-site role based at TierZero’s San Francisco headquarters, with three days a week in the office. As a founding member, direct collaboration with the CEO, CTO, and early customers shapes the direction of both product and systems. The work spans hands-on development and close engagement with users and leadership. What You Will Do Design and build intelligent AI systems to analyze large volumes of unstructured data. Deliver full-stack features based on real user feedback. Improve the product experience so AI agents are both reliable and easy for engineers to use. Develop systems that automatically evaluate LLM outputs and advance agentic reasoning using self-play and feedback loops. Create machine learning pipelines, including data ingestion, feature generation, embedding stores, retrieval-augmented generation (RAG), vector search, and graph databases. Prototype with open-source and new LLMs, comparing their strengths and weaknesses. Build scalable infrastructure for long-running, multi-step agents, with attention to memory, state, and asynchronous workflows. What We Look For Over five years of relevant professional or open-source experience. Comfort working in environments with uncertainty and evolving challenges. Strong product focus and a drive for customer satisfaction. Interest in large language models (LLMs), Model Control Planes (MCPs), cloud infrastructure, and observability tools. Previous startup experience is a plus. Location This position is based in San Francisco. Expect to work on-site three days per week at TierZero’s HQ.

TierZero builds tools that help engineering teams deliver and manage code efficiently. The platform enables quicker incident response, clearer operational visibility, and shared knowledge among engineers. Backed by $7 million from investors like Accel and SV Angel, TierZero supports clients such as Discord, Drata, and Framer as they strengthen infrastructure for AI-driven work. This in-person role is based at TierZero's San Francisco headquarters, with a hybrid schedule requiring three days onsite each week. As a founding member of the technical staff, work directly with the CEO, CTO, and customers to influence the direction of TierZero’s core products and systems. The position calls for flexibility as priorities shift and close collaboration across the company. What you will do Design and develop AI systems that handle large volumes of unstructured data. Build full-stack product features, informed by direct feedback from users. Enhance the product so agents are intelligent, reliable, and easy for engineers to use. Create systems to automatically evaluate outputs from large language models and improve agentic reasoning through self-play and feedback. Construct machine learning pipelines, including data ingestion, feature creation, embedding stores, retrieval-augmented generation (RAG) pipelines, vector search, and graph databases. Experiment with open-source and emerging large language models to compare different approaches. Develop scalable infrastructure for long-running, multi-step agents, including memory, state management, and asynchronous workflows. Requirements Interest in working with large language models, managed cloud platforms, cloud infrastructure, and observability tools. At least 5 years of professional experience or significant open-source contributions. Comfort with shifting priorities and tackling new technical problems. Strong product focus and commitment to customer outcomes. Openness to learning from a team with a track record of delivering over $10 billion in value. Ability to work onsite in San Francisco three days per week. Bonus: Experience in a startup setting and familiarity with startup dynamics.

tierzero is looking for a Founding Member of Technical Staff to help shape the direction of its technology from the ground up. This role is based at the company's San Francisco headquarters. Role overview As an early technical hire, you will work closely with engineers and product managers to build new products and features. The work centers on designing, coding, and delivering software solutions that address client needs and support tierzero's growth. Impact Contributions in this role will directly influence the company's future. The team values initiative and hands-on problem solving, giving each member a chance to make a visible difference in how the company evolves. Collaboration This position involves regular collaboration with a small, focused team. Input and ideas from every member help guide product direction and technical decisions.

tierzero seeks a Founding Member of Technical Staff to play a key role in building the company’s technology from the earliest stages. This position is based at the San Francisco headquarters and offers the chance to collaborate directly with founders and engineers. Role overview As an early team member, you will help design and develop new products and systems. The work involves close collaboration with others in the office, shaping both the technical direction and the culture of the engineering team. What you will do Develop core technology in partnership with founders and engineers Contribute ideas and code that guide the evolution of tierzero’s products Help define engineering standards and establish best practices Location This position is based onsite at the San Francisco HQ.

About Liquid AIOriginating from MIT CSAIL, Liquid AI specializes in creating versatile AI systems that operate efficiently across various platforms, from data center accelerators to on-device hardware, focusing on low latency, minimal memory consumption, privacy, and dependability. Our collaborations extend across industries including consumer electronics, automotive, life sciences, and financial services. As we undergo rapid expansion, we are on the lookout for outstanding individuals to join our journey.The OpportunityThe Vision-Language Models (VLM) team is dedicated to developing cutting-edge vision-language models that function seamlessly on devices, adhering to stringent latency and memory requirements without compromising quality. Having already launched four premier models, we are excited about what lies ahead.This team is responsible for the complete VLM pipeline, encompassing research on novel architectures, training algorithms, data curation, evaluation, and deployment. You will be part of a dedicated, hands-on team that directly engages with models and works closely with our pretraining, post-training, and infrastructure teams. Your success will be gauged by the performance of the models we deliver.

Anthropic is looking for a Research Engineer focused on model evaluations. This position involves research and development to assess and strengthen the performance of AI models. Teams are based in San Francisco and New York City, and the role supports remote work with required travel. Key responsibilities Design and implement evaluations for Anthropic's AI models Collaborate with team members to enhance model performance Contribute to research that pushes the boundaries of AI systems Location Remote-friendly (travel required) San Francisco, CA New York City, NY

Overview: Due to the increasing market demand and a robust six-month product roadmap, Listen Labs is expanding its engineering team. We seek a technically adept individual (our team includes three IOI medalists) who is eager to contribute to a product that is revolutionizing corporate decision-making. If you are passionate about solving intricate problems from start to finish, we invite you to connect with us.About Listen LabsListen Labs is an innovative AI-driven research platform that empowers teams to swiftly extract insights from customer interviews in hours rather than months. Our technology enables clients to analyze conversations, identify recurring themes, and expedite informed product decisions.Company Highlights:Exceptional Team: Composed of seasoned entrepreneurs (with prior AI exits), co-founders, and experts from leading firms such as Jane Street, Twitter, Stripe, Affirm, Bain, Goldman Sachs, and more, our team is built on a foundation of excellence.Rapid Growth: We are a dynamic team of 40, supported by Sequoia, achieving a remarkable growth trajectory from $0 to $14 million run-rate in less than a year. We prioritize speed, craftsmanship, and collaboration with individuals who embrace ownership.Impressive Traction: We have seen rapid growth across various sectors, securing enterprise clients such as Google, Microsoft, Nestlé, and P&G.Outstanding Performance: Our industry-leading win rate is a direct result of our uniquely differentiated product.Market Validation: We consistently attract customers across every segment, often landing six-figure deals that lead to quick expansions.Viral Product: Our interviews are shared with tens of thousands of viewers, driving product-led growth, organic expansion, and daily inquiries from Fortune 500 companies.Technical Challenges:Research Agent Development: Unlike traditional software purchases, hiring McKinsey involves gaining insights and execution expertise. We are building Listen Labs with that mindset — an AI agent that understands our platform and best research practices, assisting users in project setup, interview execution, and response analysis.Human Database Creation: A core value proposition is our capability to connect users with specific demographics. We are developing a database of millions of individuals, continually enhancing our understanding of user needs as they engage with Listen Labs.

Technical Staff Member in Applied AIAbout the OpportunityWe are seeking a highly skilled Technical Staff Member specializing in generative modeling to bridge the gap between our advanced models and the clients who rely on them. You will collaborate with a diverse team of machine learning experts, protein engineers, and biologists to revolutionize biological control and disease treatment. Your role will involve gaining a comprehensive understanding of our proprietary generative models and leveraging that expertise to deploy, adapt, and optimize these models in client environments, particularly within the pharmaceutical and biotech industries.This hybrid position requires a research-oriented mindset to deeply understand our models, paired with the communication skills necessary to translate that knowledge into production systems that yield scientific value for our collaborators.About UsAt Latent Labs, we are pioneering frontier models that decode the fundamentals of biology. Our ambitious goals are driven by curiosity and a commitment to scientific excellence. Prior to founding Latent Labs, our team co-developed DeepMind's Nobel Prize-winning AlphaFold, innovated latent diffusion, and created groundbreaking lab data management systems along with high-throughput protein screening platforms. Here, you will work alongside some of the brightest minds in generative AI and biology.We value interdisciplinary collaboration, continuous learning, and teamwork. Our team offsites foster a culture of trust and connection between our London and San Francisco offices. We are looking for innovators who are passionate about solving complex problems and making a positive global impact. Join us on our ambitious mission.Your QualificationsExpertise in Machine Learning: You are a proficient ML researcher with a strong background in generative modeling, evidenced by your contributions to notable open-source projects, impactful product launches, or significant publications in leading venues such as NeurIPS, ICML, ICLR, or Nature. You possess a deep understanding of generative model architectures, training dynamics, and inference behavior.Proficient ML Developer: You produce robust, tested, and maintainable ML code. Your experience includes using version control and code review systems. You are adept at rapid prototyping while also being able to write elegant production code. Additionally, you have experience in building systems that deploy large models via APIs and executing inference tasks in production environments.

At Composio, we are developing advanced infrastructure that enables agents to seamlessly interact with essential work tools such as GitHub, Gmail, Notion, Salesforce, and more. Our dedicated team of engineers is committed to tackling challenges ranging from contextual understanding to search functionalities, ensuring we provide an exceptional bridge between your agents and their tools.Having secured a $25M Series A funding from Lightspeed, alongside prominent angel investors like Guillermo Rauch (CEO of Vercel), Dharmesh Shah (CTO of HubSpot), and Gokul Rajaram, we have experienced remarkable growth, tripling our ARR at the start of this year. Our clientele includes notable names from Y Combinator cohorts to Wabi, Glean, Zoom, and beyond.Your RoleEnhance the experience of teams utilizing our platform by refining our core APIs and SDK.Create intuitive interfaces for both frontend and SDK applications.Take ownership of product development from concept through to production.Collaborate closely with customers to cultivate their loyalty while enhancing the product.Craft clear and concise documentation.

As a Technical Staff Member specializing in Machine Learning, you will:Engage in the complete development lifecycle of innovative large-scale deep learning models.Curate datasets, architect solutions, implement algorithms, and train and assess models to enhance our offerings.Work collaboratively with engineers and researchers to convert groundbreaking research into real-world applications.Join us at a pivotal time, take on diverse roles, and contribute to building transformative products from the ground up!

Overview: Join Listen Labs as we respond to a surge in market demand with an ambitious 6-month product roadmap. We are expanding our engineering team and are on the lookout for a highly skilled technical expert (our current team includes three IOI medalists) who is eager to build a transformative product that reshapes decision-making for businesses. If you have a passion for solving intricate problems from start to finish, we want to connect with you.About Listen LabsListen Labs is an AI-driven research platform designed to help teams quickly extract insights from customer interviews in a matter of hours rather than months. We empower our clients by enabling them to analyze conversations, identify key themes, and make faster, more informed product decisions.Why Work with Us?Exceptional Team: Founded by seasoned entrepreneurs with a successful AI exit, along with talent from renowned companies such as Jane Street, Twitter, Stripe, Affirm, Bain, and Goldman Sachs, our team boasts impressive credentials including IOI and ICPC backgrounds.Rapid Growth: As a 40-person team backed by Sequoia Capital, we have achieved a remarkable growth trajectory, scaling from $0 to a $14 million run-rate in less than a year. We prioritize craftsmanship and thrive on collaboration with individuals who take ownership.Impressive Traction: We are experiencing rapid growth across various sectors, securing enterprise clients such as Google, Microsoft, Nestlé, and Procter & Gamble.Proven Performance: We maintain an industry-leading win rate driven by our uniquely differentiated product.Market Validation: We consistently attract customers from diverse segments, achieving six-figure contracts that facilitate quick expansions.Viral Product: Our interviews reach tens of thousands of viewers, promoting product-led growth, organic expansion, and daily interest from Fortune 500 companies.Technical Challenges Await:Research Agent Development:Unlike traditional software purchases, hiring McKinsey offers valuable opinions, expertise, and execution. We aim to provide users with an AI agent that possesses complete knowledge about our platform and best research practices, assisting them in project setup, interview conduction, and response analysis.Human Database Creation:One of our core offerings is the ability to identify target users effectively (e.g., "power users of ChatGPT and Excel"). We are in the process of building a comprehensive database that connects users with the insights they need.

About tierzero tierzero builds tools that help engineering teams manage production code with stronger incident response, better operational visibility, and collaborative knowledge sharing. Companies like Discord, Drata, and Framer use tierzero to support their infrastructure in an AI-driven landscape. Backed by $7 million from investors including Accel and SV Angel, tierzero is growing quickly from its San Francisco headquarters. Role Overview: Founding Member of Technical Staff This is a hands-on role shaping tierzero’s core product and systems from the ground up. The founding technical team works closely with the CEO, CTO, and early customers to solve real engineering challenges. The position is based in San Francisco, with a hybrid schedule: three days each week in the office. What You’ll Do Design and build intelligent AI systems that process large volumes of unstructured data Deliver full-stack features informed by real-time user feedback Improve usability so AI agents are both effective and trustworthy for engineers Develop systems for automated evaluation of LLM outputs, including feedback loops and self-play Construct machine learning pipelines for data ingestion, feature generation, embedding storage, retrieval-augmented generation (RAG), vector search, and graph databases Prototype with open-source LLMs to understand their strengths and weaknesses Create scalable infrastructure for complex, multi-step agents, focusing on memory, state management, and asynchronous workflows Who We’re Looking For 5+ years of professional experience or significant open-source contributions Interest in LLMs, MCPs, cloud infrastructure, and observability tools Comfort working in changing, ambiguous situations Product-focused and customer-first mindset Experience learning from and collaborating with engineers from diverse backgrounds Bonus: Previous experience in a startup setting Work Location Hybrid schedule: three days per week in-person at the San Francisco HQ.

About tierzero tierzero helps engineering teams build and deploy code with greater speed and operational clarity in an AI-driven world. The company focuses on improving incident response, operational visibility, and knowledge sharing for engineers. Backed by $7 million in funding from investors like Accel and SV Angel, tierzero supports large-scale systems for clients such as Discord, Drata, and Framer. Role Overview: Founding Member of Technical Staff This role is based at tierzero's San Francisco headquarters. In-person work is required three days a week. As a founding member of the technical team, you will help design and build core products and systems from the ground up. Collaboration is central: expect to work closely with the CEO, CTO, and customers. Projects span a wide range of technical challenges and product areas. What You Will Do Design and implement intelligent AI systems that process and reason over large volumes of unstructured data. Develop full-stack features, incorporating direct feedback from users. Improve the product experience so intelligent agents are practical and reliable for engineers. Create systems that automatically evaluate LLM outputs and refine agent reasoning using self-play and feedback loops. Build machine learning pipelines covering data ingestion, feature generation, embedding stores, RAG pipelines, vector search, and graph databases. Prototype and experiment with open-source and advanced LLMs to weigh different approaches. Set up scalable infrastructure for long-running, multi-step agents, including memory management, state handling, and asynchronous workflows. What We Look For At least 5 years of professional or open-source experience in a relevant technical field. Comfort working in a setting that changes and evolves quickly. Strong product focus and an understanding of customer needs. Interest in LLMs, MCPs, cloud infrastructure, and observability tools. Ability to learn from and collaborate with engineers who have delivered over $10 billion in value. Commitment to working onsite in San Francisco three days per week. Startup experience is a plus.

Shape the Future with AI InnovationAt asari.ai, we are on a mission to empower individuals to create intricate systems and tackle the world's most challenging problems with the aid of advanced AI agents that are both scalable and reliable.Our team boasts a history of publishing award-winning AI research and is supported by elite investors such as Eric Schmidt, Caltech, Jeff Dean, and JP Millon.We thrive in a fast-paced environment, leveraging first-principles thinking and purposeful building. We believe that exceptional outcomes arise when individuals take ownership, grow collectively, and share both the hurdles and victories.Your RoleDesign, train, and assess hybrid AI systems that excel at scale while making optimal trade-offs.Develop scalable data processing and machine learning frameworks.Address min-max challenges: maximizing output while minimizing resources.Enhance our productivity by eliminating operational and tooling constraints.Ideal Candidate ProfileYou find motivation in resolving complex real-world challenges.You possess a proven background in a technical field such as machine learning, computer science, physics, or mathematics.You demonstrate strong programming skills (in Python, C++) and mathematical proficiency.You exhibit robust conceptual and structured thinking abilities.You communicate clearly and effectively, both verbally and in writing.You are eager and capable of rapid learning.You embody a team-oriented spirit.You can independently organize, plan, prioritize, and execute tasks.You are driven by excellence, ownership, and a proactive mindset.Preferred QualificationsExperience with open-source projects, published research, or demonstrated expertise in machine learning.Familiarity with applying deep learning, reinforcement learning, unsupervised learning, and related techniques to large-scale problems.Experience with distributed computing and managing large datasets.Compensation and BenefitsAttractive salaryEquity optionsComprehensive health insurance (100% covered), dental (90% covered), and vision (90% covered). Dependent coverage is at 50%.

Technical Staff MemberMirendil is a pioneering technology company dedicated to addressing fundamental challenges that propel significant advancements in science and technology. Our primary mission is to democratize access to cutting-edge AI research and development across various scientific fields. We believe that accelerating scientific discovery is one of the most impactful ways to enhance humanity's future, with AI playing a crucial role in achieving this vision.We are in the process of establishing a leading AI research company, developing our own models from the ground up. Our focus encompasses model training, reinforcement learning, reasoning systems, and the infrastructure necessary for large-scale experiments. Our team comprises accomplished researchers and engineers from esteemed organizations such as Anthropic, Google DeepMind, xAI, OpenAI, Microsoft, Apple, and MIT.Position OverviewWe are seeking skilled engineers and researchers to join us as Members of Technical Staff.This role is designed to be flexible and open-ended. Depending on your expertise and interests, you may engage in:Enhancing and training advanced AI modelsDeveloping reinforcement learning and reasoning systemsBuilding infrastructure for extensive experimental projectsCreating systems to automate or expedite research workflowsIf you are passionate about tackling ambitious challenges at the crossroads of AI, research, and scientific innovation, we would love to connect with you.

At teameigen, we are a dynamic group of innovators, engineers, and storytellers, supported by renowned venture capitalists and the visionaries behind some of the world’s most cherished consumer products, as well as influential figures from film, music, and television.We seek passionate builders with a keen sense of product development to advance applied emotional intelligence and create the technological frameworks that will define consumer technology in the next decade.As a member of our technical staff, you will:Independently tackle complex and ambiguous challenges from start to finish.Engage across the entire technology stack with a product-first mindset rather than a tech-centric approach.Collaborate with writers, artists, and other creatives to enhance the emotional and aesthetic experience of our product.Demonstrate relentless urgency and a strong work ethic in all projects.