About the job
About Us
Braintrust is revolutionizing AI observability. By merging evaluation and observability into a single workflow, we empower developers with the insights necessary to understand AI performance in real-world applications, enabling ongoing enhancements.
Our platform is trusted by leading companies like Notion, Stripe, Zapier, Vercel, and Ramp to streamline model comparisons, prompt testing, and regression detection — transforming production data into refined AI with each iteration.
Role Overview
We are seeking a skilled Customer Solutions Architect to join our dynamic Field Engineering team. This pivotal position is highly technical and client-focused, aimed at assisting teams in deploying and managing Braintrust in their production environments.
You will collaborate with platform, DevOps, and machine learning infrastructure teams within pioneering AI-centric organizations to ensure Braintrust operates seamlessly—whether in our managed cloud or within their private infrastructures. Your expertise will be essential in evaluating and monitoring LLM-driven applications, enabling effective evaluation workflows through robust infrastructure support. You will assist with deployments, troubleshoot multi-layer issues, and advocate best practices for scalability, security, and maintainability.
This role combines solution design, proactive support, and infrastructure advisory, making it perfect for someone passionate about resolving real-world challenges through coding, systems knowledge, and a strong customer-centric approach.
Your Responsibilities
Customer Enablement & Support
Serve as a trusted advisor for customers deploying and scaling Braintrust in production settings.
Lead onboarding and implementation efforts, assisting in the setup of customer-hosted infrastructures, configuration of observability pipelines, evaluation workflows, and integration with third-party applications.
Troubleshoot customer issues across various layers (e.g., SDKs, cloud infrastructure, model outputs), coordinating with product and engineering teams as necessary.
Manage proactive health checks, upgrade support, and reliability best practices for clients running self-hosted deployments.
Solution Design & Implementation
Design and document comprehensive customer evaluation pipelines utilizing Braintrust SDKs and APIs.
Create technical artifacts (scripts, templates, notebooks) that facilitate onboarding and enhance our customer support capabilities, or showcase platform functionalities.
Advise on architectural decisions regarding LLM evaluation, prompt versioning, and structural best practices.

