companyCanvas Medical logo

Applied AI Software Engineer

Canvas MedicalSan Francisco, CA / Remote
Remote Full-time

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Unlock Your Potential

Generate Job-Optimized Resume

One Click And Our AI Optimizes Your Resume to Match The Job Description.

Is Your Resume Optimized For This Role?

Find Out If You're Highlighting The Right Skills And Fix What's Missing

Experience Level

Experience

Qualifications

To be successful in this role, candidates should possess:Extensive experience with LLM-based agent evaluations at scale. Proficiency in designing rigorous evaluation methodologies. A strong understanding of AI model fine-tuning and performance metrics. Excellent analytical skills and attention to detail. Ability to work collaboratively across multiple teams. Strong communication skills to effectively convey complex concepts to non-technical audiences.

About the job

Canvas Medical is at the forefront of transforming healthcare with our cutting-edge electronic medical records (EMR) and payments platform. Our mission is to create a seamless environment where developers and clinicians can collaborate effectively to tackle the most pressing challenges in healthcare. Backed by some of the world's leading technology investors, including those who have funded notable health tech innovators like GoodRx, Oscar Health, and Hims & Hers Health, we are on a journey to revolutionize the way healthcare is delivered.

The Role

We are seeking a talented Applied AI Software Engineer to spearhead the evaluation of our developing agents and the post-deployment fleet of agents in Canvas, aimed at automating tasks for our clients. You will leverage state-of-the-art foundation model inference and fine-tuning APIs, along with our server-side SDK which is equipped with a wealth of tools and contextual information essential for optimal agent performance. Your responsibilities will include designing and executing comprehensive evaluation experiments that assess performance, safety, and reliability across various clinical, operational, and financial scenarios.

This position is perfect for individuals with substantial experience in evaluating LLM-based agents on a large scale. You will be tasked with creating precise unit evaluations and end-to-end assessments, establishing expert-determined ground truth outcomes, and managing iterations across model variants, prompts, tool utilization, and context window configurations. Your insights will play a critical role in guiding model selection, fine-tuning, and go/no-go decisions for AI features utilized in production environments.

Collaboration is key; you will work closely with product managers, machine learning engineers, and clinical informatics teams to ensure our AI agents are not only effective but also reliable and robust under real-world healthcare constraints. Additionally, you will partner with technical product marketers and developer advocates to communicate the unique value propositions of Canvas's AI agents to our broader developer community and the market.

About Canvas Medical

At Canvas Medical, we empower healthcare professionals with innovative technology solutions that streamline operations and enhance patient care. Our commitment to excellence and collaboration drives us to develop tools that foster meaningful interactions between developers and clinicians, ultimately transforming the healthcare landscape.

Similar jobs

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.