Applied AI Engineer

Join Scale AI as a passionate and technically adept AI Research Engineer within our Enterprise Evaluations team. This pivotal role is integral to our goal of providing the industry's leading Generative AI Evaluation Suite. You will actively contribute to the foundational systems that guarantee the safety, dependability, and ongoing enhancement of LLM-driven workflows and agents for enterprise clients. The perfect candidate will possess a robust understanding of large language models, a fervor for addressing intricate evaluation dilemmas, and the ability to excel in a fast-evolving research atmosphere. We seek an engineer who can innovate, remains informed about the latest studies in AI evaluation, and is enthusiastic about incorporating cutting-edge research concepts into our workflows to create top-tier evaluation systems.

Join Our Innovative Team at ArcadeAt Arcade, we are revolutionizing the way physical products are created with our cutting-edge AI platform. We empower individuals to turn their creative ideas into tangible products seamlessly, utilizing natural language and generative AI. Our mission is to democratize product design, making it as effortless as sharing a post online.Backed by a remarkable $42M in funding from industry-leading investors including Reid Hoffman and Ashton Kutcher, our company is a rising star in the tech landscape. Guided by our founder Mariam Naficy and a team steeped in AI and design expertise, we are at the forefront of a new frontier that merges AI, personal expression, and on-demand manufacturing.Your Role as an Applied AI EngineerWe are on the lookout for an Applied AI Engineer to enhance our generative AI capabilities. This position combines hands-on model development with the integration of advanced AI techniques into our production systems. You will collaborate with diverse teams to conduct research, experiment with models, and implement AI-driven products.

About UsAt Applied Compute, we are pioneering the development of Specific Intelligence for enterprises, creating agents that continuously learn from a company’s processes, data, expertise, and objectives. Our mission is to bridge the gap between isolated AI capabilities and their effective application within real business environments. Traditional AI systems often fall short as they lack the ability to adapt based on feedback. Our innovative continual learning layer captures context, memory, and decision-making processes across the enterprise, enabling specialized agents to engage in meaningful work.What Excites Us: We operate at the exciting intersection of product development and cutting-edge research. Our product team designs the platform that empowers a new generation of digital coworkers, while our research team drives advancements in post-training and reinforcement learning to enhance user experiences. As an applied research engineer, you will work directly with clients to implement models in production, combining robust product development with deep research insights to facilitate AI integration in enterprises.Meet Our Team: Our diverse team consists of engineers, researchers, and operators, many of whom are former founders. We have previously built reinforcement learning infrastructure at OpenAI, established data foundations at Scale AI, and contributed to significant systems at companies like Together, Two Sigma, and Watershed. We collaborate with Fortune 50 clients, including DoorDash, Mercor, and Cognition, and are proud to be backed by reputable investors such as Benchmark, Sequoia, and Lux.Who Thrives Here: We seek individuals who are passionate about applying innovative research and complex systems to solve real-world challenges. You should feel comfortable navigating new environments rapidly—be it a fresh codebase, a client’s data architecture, or an unfamiliar problem domain. A genuine enjoyment for customer interaction, empathy, and a deep understanding of their operational workflows are essential. Candidates with entrepreneurial backgrounds, extensive side projects, or a proven track record of end-to-end ownership typically excel in our environment.

Quizlet Inc. is looking for an Applied AI Engineer to create AI-driven features that support student learning. This position centers on developing and deploying machine learning solutions aimed at making study experiences more effective and engaging for a global user base. What you will do Design and implement machine learning models to enhance Quizlet’s educational tools Work on features that help students study more efficiently and enjoyably Locations Denver, CO New York, NY San Francisco, CA Seattle, WA

Join the Visionaries at Automat!At Automat, we are a collective of innovative technologists hailing from prestigious backgrounds at Google Creative Lab and Samsung’s innovation Think Tank. Our mission is to transform the future of business through intelligent agents that enhance operational efficiency and eventually manage enterprises autonomously. We are leaders in redefining Enterprise AI Agents and Intelligent Document Processing, creating cutting-edge tools and infrastructure to seamlessly integrate AI advancements into practical applications.We cherish curiosity, teamwork, and impactful contributions over strict credentials. If you are passionate about AI and automation, excel in dynamic and creative settings, and yearn to engage in groundbreaking projects, we invite you to apply!While this is a senior role, we believe that unique talents and perspectives can surpass traditional experience metrics. We seek an individual who can make an immediate impact from day one and contribute to our team’s learning journey.Your RoleAs an Applied AI Engineer, you will operate at the crossroads of product development, engineering, and research. You will work hand-in-hand with skilled engineers and end users to create innovative tools for our internal teams and external clientele. Your contributions will help shape technical solutions, guide strategic initiatives, and enrich our company culture.Key Responsibilities:Collaborate with product and engineering teams to gain insights into customer challengesResearch and implement real-world applications of advanced AI modelsDesign and execute impactful solutions for both internal tools and customer-facing productsInfluence engineering practices, guide product development, and foster a collaborative cultureWhat Motivates You:A passion for curiosity, creativity, and quick adaptationCreating full-stack applications that bring AI to lifeKeeping abreast of the latest trends in AI and LLM prompting techniquesChallenging the limits of what is achievable with AI agentsThriving in fast-paced, collaborative environments

About WorkOS WorkOS is revolutionizing the developer landscape by creating cutting-edge tools and APIs that empower businesses to achieve Enterprise Ready status. Our platform is the backbone for authentication, identity management, authorization, and other essential infrastructures, enabling developers to securely scale their products for large enterprises.Having recently secured a $100M Series C funding round, valuing the company at $2B, we are supported by leading investors such as Meritech, Sapphire, Greenoaks, Craft, Abstract, and Audacious. WorkOS is proud to serve many of the fastest-growing AI companies, including OpenAI, Cursor, Perplexity, Vercel, and Plaid.As AI technology evolves, WorkOS stands at the forefront of Human and Agent Authentication, Identity, and Access Control, helping businesses navigate crucial questions about agent identity and permissions. Our rapidly expanding customer base features hundreds of innovative software companies developing the next generation of enterprise-ready solutions.About the RoleWe are assembling an Applied AI team dedicated to significantly enhancing productivity across Engineering, Sales, Support, and Operations while also developing AI functionalities for our clients.As an Applied AI Engineer, you will design and deploy transformative AI systems that redefine how WorkOS develops, markets, and supports its software. You will focus on creating robust tools and workflows that are essential for daily operations, while also exploring novel AI capabilities for our users.Join a small, high-ownership team that:Identifies challenges based on measurable impact.Rapidly transitions from idea to prototype to production in days or weeks.Quickly adapts to evolving models, tools, and best practices.This role offers substantial visibility across the organization.What You’ll DoCollaborate closely with Engineering, Sales, Support, and Operations to identify high-impact internal challenges where AI can enhance efficiency or facilitate new workflows.Design and construct tools that integrate seamlessly into daily operations—agents, automations, and workflows that are reliable, observable, and easy to maintain.Leverage LLMs, embeddings, retrieval systems, and tool-calling to connect with documents, Slack, GitHub, CRM systems, analytics, support frameworks, and internal services.Transform repetitive, multi-step manual tasks into streamlined, AI-driven processes that span various applications and data sources.Stay informed about emerging models and tools, conduct focused experiments, and assist the team in continuous improvement.

About Us:At LangChain, we strive to revolutionize the accessibility of intelligent agents. Our goal is to provide a robust framework for agent engineering that empowers developers to transition from initial prototypes to production-ready AI agents that can be trusted by teams. What started as a suite of widely embraced open-source tools has evolved into a comprehensive platform designed for the building, assessment, deployment, and management of agents at scale.Currently, our tools, including LangChain, LangGraph, LangSmith, and Agent Builder, are utilized by various teams delivering tangible AI solutions across both startups and large corporations. Millions of developers depend on LangChain to empower AI initiatives at prestigious companies like Replit, Clay, Coinbase, Workday, Lyft, Cloudflare, Harvey, Rippling, Vanta, and 35% of the Fortune 500.With $125M raised in our Series B round from IVP, Sequoia, Benchmark, CapitalG, and Sapphire Ventures, we are in a phase of rapid growth, continuously innovating new products while ensuring every team member has a significant impact on our development and collaborative processes. LangChain is a place where your contributions can influence the real-world application of this transformative technology.About the Role:We are seeking talented Applied AI Engineers to assist in creating AI agents that enhance various aspects of LangChain, from Marketing and Go-To-Market strategies to Recruiting, Support, Internal Tools, and our Core Product.In this position, you will take ownership of specific problem areas, collaborating closely with relevant teams to design, construct, and deploy production-grade agents, workflows, and applications that revolutionize our operational processes. Your contributions will directly support LangChain’s vision of making intelligent, autonomous software a reality for both our internal teams and our clients. Some projects may be open source, contributing to the LangChain and LangGraph ecosystems while establishing new benchmarks for AI development practices.If you are a full-stack software engineer eager to implement AI agents in practical use cases and witness their impact on business outcomes, we invite you to apply.

Join Our Team as an Applied AI EngineerAt Rowspace, we're at the forefront of AI research and engineering, and we are looking for an Applied AI Engineer to help revolutionize how our platform uncovers vital financial insights. Your week may involve developing and deploying AI agents based on client feedback, creating evaluation frameworks for AI performance, and designing sophisticated retrieval systems to optimize AI within financial workflows.Your ResponsibilitiesIntegrate groundbreaking AI research to develop systems that comprehend financial contexts and business concepts.Construct retrieval pipelines that intelligently extract and synthesize insights from unstructured data sources such as PDFs, emails, and presentations.Develop workflows that empower AI systems to analyze financial relationships, conduct benchmarking, and perform comparative analyses.Design and maintain knowledge graphs that represent intricate financial relationships, entity connections, and market dynamics.Create evaluation frameworks to assess and enhance AI performance in investment research, due diligence, and portfolio monitoring.Implement systems ensuring traceability from AI outputs to primary data sources.Experiment with innovative LLM orchestration patterns and context management techniques tailored for financial applications.Guide the technical direction and establish best practices for AI application development at Rowspace.Engage in rapid iteration with direct customer feedback to continuously enhance AI systems based on real-world usage.Oversee the deployment and operation of production systems end-to-end, ensuring reliability, performance, and maintainability across the technology stack.

About Komodo Health We Bring Data to Life Komodo Health’s mission is to alleviate the global burden of disease. The company’s Healthcare Map integrates de-identified patient data with advanced algorithms and clinical expertise, creating a detailed view of the U.S. healthcare system. This foundation supports a suite of software applications that help partners address complex healthcare challenges, monitor patient behaviors, spot treatment trends, and identify care gaps. Komodo Health values excellence, growth, results, and enjoying the journey. The team brings together professionals from varied backgrounds who share a commitment to improving health outcomes. Role Overview: Applied AI Engineer Healthcare in the U.S. is intricate, and Komodo Health tackles these challenges through data-driven solutions. By mapping the patient journey nationwide, the company helps pharmaceutical firms, payers, and healthcare systems make decisions that improve patient care. Labs@Komodo builds AI-native platforms and systems that turn data into actionable insights. The team developed Marmot, Komodo’s AI-native product, which embeds AI into both the user interface and development workflow. By combining Komodo’s healthcare data with modern large language models (LLMs), Marmot delivers real-world insights for healthcare stakeholders. What You Will Do Design and implement end-to-end AI solutions for real products and internal tools. Work at the intersection of applied research, engineering, and product development. Integrate modern AI techniques into scalable, production-ready systems. Collaborate with product, platform, and data teams to build AI capabilities that advance healthcare data use. Location San Francisco, CA

About the Role sfcompute is hiring an Applied AI Engineer in San Francisco, CA. This position focuses on building and deploying AI models that support real business needs. The work involves using advanced technologies to create practical solutions that improve how teams operate. What You Will Do Develop and implement AI models tailored to business goals Work closely with teams across the company to understand their challenges Deliver solutions that boost productivity and efficiency

ABOUT USAt Applied Compute, we are pioneers in developing Specific Intelligence for enterprises, creating agents that learn continuously from a company’s processes, data, expertise, and objectives. Our mission is to establish a continual learning platform that captures context, memory, and decision traces throughout the organization, enabling specialized agents to perform meaningful tasks.Why Join Us: Our team operates at a unique intersection of innovation. Our product team is responsible for crafting a platform that serves as the backbone for a new generation of digital coworkers. Meanwhile, our research team explores the cutting edge of post-training and reinforcement learning to enhance product experiences. Our applied research engineers collaborate closely with clients to deploy agents effectively in real-world scenarios. This synergy of robust product development, extensive research, and direct client engagement is essential for us to revolutionize AI in the enterprise landscape.Our Team: Comprising engineers, researchers, and operations experts, our team includes many former founders with extensive experience. We have developed RL infrastructure at OpenAI, data foundations at Scale AI, and other systems at companies like Two Sigma and Watershed. We proudly serve Fortune 50 clients and are supported by top-tier investors including Kleiner Perkins, Benchmark, Sequoia, Lux, and Greenoaks.Who Thrives Here: We seek individuals passionate about utilizing cutting-edge research and complex systems to address real-world challenges. Comfort navigating diverse environments, whether it’s a new codebase, unfamiliar customer data architecture, or unexplored problem domains, is essential. Our team values genuine client engagement — listening, empathizing, and understanding the realities of work in their organizations. Those with entrepreneurial spirits, rich project experiences, or proven capabilities to manage tasks end-to-end will excel in our environment.THE POSITIONAs a Software Engineer, you will be instrumental in building the products and interfaces utilized by customers and internal teams. You will manage the entire application platform stack, from collaborative human-AI workspace systems to backend workflows orchestrating sandboxed agent sessions, and the continual learning SDK that provides engineers with oversight of the agent development lifecycle.

About ArtisanAt Artisan, we are pioneers in creating fully autonomous AI employees – not mere chatbots or copilots, but digital workers capable of performing meaningful tasks.Our flagship product, Ava, is an AI-powered Business Development Representative (BDR) utilized by hundreds of companies. Ava excels at researching leads, crafting and sending emails in clients' unique voices, managing complex outbound sequences, autonomously optimizing her performance, and even addressing objections while scheduling meetings. She is not just a tool; she is a collaborative teammate.As a proud Y Combinator W24 company, we have successfully raised over $35 million in funding and are currently achieving over $8 million in annual recurring revenue (ARR). We are now embarking on the development of Ava 2.0, which will redefine the capabilities of AI employees. The engineering challenges involved are substantial, and the scope of our project is vast.Role OverviewJoin our innovative team as the third Applied AI Engineer at Artisan! You will be instrumental in pushing the boundaries of AI employee capabilities and guiding our product's future direction.Evaluate LLMs: Select optimal models for various tasks while considering cost, latency, reliability, and accuracy.Architect Prompt Frameworks: Design agent behaviors for Ava's essential workflows such as email generation, chat interactions, meeting scheduling, prospect research, and more.Optimize Multi-Step Agent Chains: Implement retrieval-augmented generation (RAG), integrate web searches, and utilize tools like CRMs and APIs.Drive Infrastructure Decisions: Lead choices related to routing, orchestration, evaluation loops, and persistent memory for agents.Build Safety and Trust Mechanisms: Collaborate with the product team to design user guardrails, fail-safes, and success metrics.Explore Emerging Modalities: Investigate and deploy voice AI, talking head technology, and multi-modal reasoning to enhance Ava's human-like interactions.Design Autonomous Agent Workflows: Create workflows that enable strategic decision-making, real-time self-optimization, and measurable outcomes.Location: San Francisco, New York, or Remote USATeam: AIReports to: CPTO, Sam Stallings

Role Overview:As an Applied AI Engineer at Mem0, you'll lead the charge from concept to execution. Your mission is to transform abstract customer use cases into tangible proofs-of-concept that vividly demonstrate Mem0's capabilities. This entails rapid full-stack prototyping, leveraging AI tools, and rigorously experimenting with memory retrieval methodologies until the end-to-end use case is operational. Collaborating closely with Research and Backend teams, you'll clearly communicate trade-offs and deliver successful prototypes ready for production refinement.Your Responsibilities:Develop Proofs of Concept: Create comprehensive demos (UI + APIs + data) that seamlessly integrate Mem0 into customer workflows.Innovate in Memory Retrieval: Experiment with various embeddings, indexing techniques, hybrid search, re-ranking, chunking/windowing, prompts, and caching to achieve optimal task-level quality and latency.Collaborate on Prototypes: Implement groundbreaking ideas and techniques sourced from academic research, compare them against established baselines, and adopt the most effective solutions.Design Evaluation Frameworks: Establish small gold sets and lightweight metrics to assess POC success; equip demos with basic telemetry.Integrate AI Tools: Combine LLMs, vector databases, Mem0 SDKs/APIs, and third-party services into cohesive workflows.Foster Close Collaboration: Work alongside Backend teams to establish clean contracts and data models; collaborate with Research on hypotheses; share insights and future steps.Document and Handoff: Produce clear documentation, scripts, and templates to facilitate rapid production transition by Engineering.Minimum Qualifications:Full-Stack Proficiency: Experience with Next.js/React for front-end development and Python backends (FastAPI/Django/Flask) or Node.js as required.Proficient in Python and TypeScript/JavaScript; capable of building APIs, wiring data models, and deploying quick demos.Hands-on experience with the LLM/RAG stack: embeddings, vector databases, retrieval strategies, and prompt engineering.Proven ability to rapidly prototype: transforming ideas into demos within days, not months; clear documentation of outcomes and trade-offs.Capacity to design small, impactful evaluations for use cases (quality + latency) and iterate based on data-driven evidence.Outstanding communication skills with Research and Backend teams; ability to produce crisp specifications, readable code, and honest status updates.

Role Overview Paraform is hiring an Applied AI Engineer in San Francisco. This role focuses on building and deploying AI systems that directly serve real users. The ideal candidate brings 2-5 years of experience, a strong grasp of modern LLM-based technologies, and a track record of turning advanced models into reliable product features. Success in this role depends on sound product sense and the ability to weigh trade-offs between LLM and traditional machine learning approaches. Experience with LLM-powered applications, retrieval systems, agentic workflows, or automation is valuable. Familiarity with classic ML techniques, such as ranking, recommendation, or classification, will help in designing hybrid systems that balance performance, cost, and reliability. What You Will Do Design and build AI systems to improve matchmaking, ranking, and automation in the Paraform marketplace. Develop LLM-driven features, including retrieval pipelines and agentic workflows, to streamline recruiter and company interactions. Own systems end-to-end: from data pipelines and model design to deployment, monitoring, and iteration in production. Work closely with product managers, ML engineers, and full-stack teams to deliver AI capabilities that shape marketplace outcomes. Create evaluation frameworks to measure real-world performance, reliability, and business impact, not just offline metrics. Set best practices for building and maintaining production AI systems, balancing model quality, cost, latency, and maintainability. Advance the integration of AI into product experiences across the platform. What We Look For 2-5 years of experience at an AI-focused startup (Series A through D). Background working on products with a broad user base, beyond single-enterprise deployments. Proficient in Python and Typescript. Experience developing agentic systems that drive measurable business or user outcomes. Comfort with ambiguity and building in 0 to 1 environments. Ability to communicate technical trade-offs clearly to non-technical stakeholders.

Applied AI Engineer (On-Site, San Francisco)About UsAt Delve, we are pioneering an AI-native platform that revolutionizes compliance processes, turning cumbersome manual tasks into seamless automated workflows.As the fastest growing compliance technology firm in the industry, we empower hundreds of companies, such as Lovable, 11x, and WisprFlow, to reclaim countless hours by eliminating compliance busywork and allowing them to concentrate on their core mission: providing secure support to their customers.Our dynamic team consists of passionate builders—former founders, operators, and engineers—dedicated to addressing real challenges faced by our clients. Our backgrounds span prestigious institutions like Stanford, MIT, Berkeley, and OpenAI, and we are supported by top-tier investors, including Insight Partners, General Catalyst, and Y Combinator. Following our recent $32 million Series A funding, we are in an exciting phase of rapid expansion!Step by step, we are committed to eradicating busywork for humanity.Role OverviewAs an Applied AI Engineer, you will be crucial in translating cutting-edge AI research into practical applications. Your primary focus will be on converting state-of-the-art techniques into market-ready products. You will work closely with product engineers to design, implement, and roll out AI-powered features across various stages. This position offers immense impact, allowing you to lead projects from conception to production, influencing both our products and technical roadmap.You will collaborate with a top-tier team of engineers with research expertise to develop innovative, scalable systems. In our fast-paced startup environment, you’ll have the chance to shape our technical direction, share ideas across the entire stack, and quickly advance into leadership roles as we grow. We prioritize first-principles thinking and a proactive attitude—providing you with the autonomy to experiment and iterate towards optimal solutions.What You Will DoDrive AI Innovation: Research, prototype, and develop cutting-edge AI systems that enhance our products and solve core technical challenges.End-to-End Development: Design, implement, and deploy AI pipelines and systems—from evaluation design and benchmarking to product integration—ensuring high performance, reliable observability in production, and comprehensive safety and transparency.

RoleAs a Software Engineer specializing in Applied AI at OpenEvidence, you will be instrumental in developing comprehensive systems that harness cutting-edge AI models to create impactful, user-centric products.We seek exceptional builders who thrive outside conventional boundaries. Our engineers engage across various projects and products, taking ownership to maximize their impact.About UsOpenEvidence stands as the leading medical AI platform globally.In just over a year, over 40% of US clinicians have adopted our solution, driven by product-led growth and word-of-mouth recommendations. We are a $12 billion company with a talented 30-person engineering team hailing from prestigious institutions like MIT, Harvard, and Stanford. We believe that transformative products emerge from a small cadre of exceptional, autonomous builders who are empowered to take ownership and drive swift progress. We are expanding our team to capture an extraordinary opportunity to establish a benchmark platform for medical AI.If you are a top-tier engineer or scientist eager to innovate at the forefront of technology and deliver meaningful results that affect millions of lives, we want to connect with you.CultureWe believe that work should reflect a world-class commitment. Innovating from 0 to 1 and scaling from 1 to 1000 is akin to a professional sport, and we set the bar at uncompromising excellence. We understand that groundbreaking technologies are only possible through complete ownership. Significant achievements arise when individuals take decisive action.Who are you?If you are seeking a typical 9-5 job or merely wish to write papers, this position is not for you. If you are ready to dive in, get hands-on, face challenges head-on, and create something impactful that can reach millions and generate significant revenue, this role might be your perfect fit.The ideal candidate is a brilliant builder—intelligent, driven, resourceful, independent, meticulous, motivated, hardworking, and humble. Does that sound rare? It is indeed rare; we currently have only 30 such individuals, and we are eager to find more.LocationAll full-time engineering roles require in-person attendance five days a week in San Francisco or Miami.

ABOUT BASETENAt Baseten, we empower the leading AI companies of today, including Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer, by providing essential inference capabilities. Our unique blend of applied AI research, adaptable infrastructure, and intuitive developer tools enables innovators at the cutting edge of AI to seamlessly transition advanced models into production. With our recent success in securing a $300M Series E funding round, backed by notable investors such as BOND, IVP, Spark Capital, Greylock, and Conviction, we're on an exciting growth trajectory. Join our team and contribute to the platform that engineers rely on to launch AI-driven products.THE ROLEAs an Applied AI Inference Engineer at Baseten, you'll collaborate closely with clients to design, develop, and implement high-performance AI applications using our platform. You will guide customers through the entire process, from initial concept to deployment, transforming vague business objectives into dependable, observable solutions that meet defined quality, latency, and cost metrics.This position is ideal for innovative engineers eager to gain insight into how modern organizations scale AI adoption. You will thrive if you enjoy a multifaceted role that intersects product development, software engineering, performance optimization, and direct customer engagement.It’s essential to note that this position requires hands-on coding and software development, while also encompassing elements of product management, technical customer success, and pre-sales engineering.EXAMPLE INITIATIVESExplore insights from our Forward Deployed Engineering team through these blog posts: Forward Deployed Engineering on the frontier of AIThe fastest, most accurate Whisper transcriptionDeploy production-ready model servers from Docker imagesDeploy custom ComfyUI workflows as APIs...

Role Overview Arcade is looking for a Senior Applied AI Engineer in the San Francisco Bay Area. This position focuses on building solutions that use artificial intelligence to improve user experiences across Arcade’s products. What You Will Do Work with cross-functional teams to design and develop AI-driven features. Apply expertise in AI algorithms and software development to solve challenging problems. Contribute to the direction and impact of Arcade’s AI initiatives. Who We’re Looking For Extensive experience with artificial intelligence technologies. Strong background in software development and AI algorithms. Comfort working collaboratively with teams from different disciplines.

Canvas Medical is at the forefront of transforming healthcare with our cutting-edge electronic medical records (EMR) and payments platform. Our mission is to create a seamless environment where developers and clinicians can collaborate effectively to tackle the most pressing challenges in healthcare. Backed by some of the world's leading technology investors, including those who have funded notable health tech innovators like GoodRx, Oscar Health, and Hims & Hers Health, we are on a journey to revolutionize the way healthcare is delivered.The RoleWe are seeking a talented Applied AI Software Engineer to spearhead the evaluation of our developing agents and the post-deployment fleet of agents in Canvas, aimed at automating tasks for our clients. You will leverage state-of-the-art foundation model inference and fine-tuning APIs, along with our server-side SDK which is equipped with a wealth of tools and contextual information essential for optimal agent performance. Your responsibilities will include designing and executing comprehensive evaluation experiments that assess performance, safety, and reliability across various clinical, operational, and financial scenarios.This position is perfect for individuals with substantial experience in evaluating LLM-based agents on a large scale. You will be tasked with creating precise unit evaluations and end-to-end assessments, establishing expert-determined ground truth outcomes, and managing iterations across model variants, prompts, tool utilization, and context window configurations. Your insights will play a critical role in guiding model selection, fine-tuning, and go/no-go decisions for AI features utilized in production environments.Collaboration is key; you will work closely with product managers, machine learning engineers, and clinical informatics teams to ensure our AI agents are not only effective but also reliable and robust under real-world healthcare constraints. Additionally, you will partner with technical product marketers and developer advocates to communicate the unique value propositions of Canvas's AI agents to our broader developer community and the market.