DevOps Engineer at litellm | San Francisco

Senior Backend EngineerJoin LiteLLM, the leading AI Gateway trusted by industry giants including Adobe, Netflix, and NASA. Our platform empowers developers with secure and reliable access to Large Language Models (LLMs) and related services. We are currently seeking a dedicated Senior Backend Engineer to contribute to the development of robust guardrails and observability tools at scale.About The RoleIn this role, you'll take ownership of our guardrail and logging implementations. You will oversee backend code to ensure all guardrail interactions are logged accurately, user errors are communicated effectively, and that our observability tools function seamlessly under high traffic conditions. Your meticulous attention to detail regarding latency metrics, logging traceability, and backend guardrail registration will significantly bolster user confidence in our security and compliance capabilities.ResponsibilitiesDevelop and enhance our product to maximize performance, reliability, and ongoing improvements.Ensure guardrail and policy enforcement calls (e.g., applyguardrail) are accurately logged and traceable through our SpendLogs and relevant database tables.Design and implement CPU-level guardrails to mitigate common attacks on LLM APIs, MCP servers, and Agents.Identify and resolve silent failure points in guardrail creation, registration, and policy application—ensuring robust error handling and transparency for end users.Collaborate with observability tools such as Datadog, Splunk, Prometheus, and OpenTelemetry to maintain accurate, configurable, and effective monitoring and logging for backend systems.Enhance observability integrations to manage over 1 billion requests per month with minimal latency and no memory leaks due to Prometheus metrics cardinality.Work cross-functionally on backend engineering priorities including performance, reliability, and security enhancements.What We’re Looking ForBachelor’s or Master’s degree in Computer Science or a related field.4+ years of experience with Python and backend frameworks (e.g., FastAPI, Flask).Strong understanding of logging best practices, error handling, and secure backend development.Familiarity with monitoring, logging, or metrics tools.

Backend Engineer - New Graduate OpportunityJoin LiteLLM, the leading AI Gateway, trusted by renowned organizations such as Adobe, Netflix, and NASA. Our innovative platform provides developers with secure and reliable access to Large Language Models (LLMs) and related services. We are seeking a passionate Backend Engineer (New Grad) to contribute to building robust guardrails and observability tools at an extensive scale.Role OverviewIn this role, you'll be instrumental in enhancing our guardrails and logging mechanisms. You will take ownership of the backend code ensuring that all guardrail calls are accurately logged, errors are made visible to users, and our observability tools are effective under high-traffic conditions. Your meticulous attention to detail in latency metrics, logging traceability, and backend guardrail registration will significantly influence user trust in our security and compliance features.Key ResponsibilitiesEnsure all guardrail and policy enforcement calls (e.g., applyguardrail) are logged and traceable within our SpendLogs and relevant database tables.Proactively identify and resolve silent failures in guardrail creation, registration, and policy application, ensuring robust error handling and clarity for end-users.Collaborate with observability integrations, including Datadog, Splunk, Prometheus, and OpenTelemetry, to maintain effective monitoring and logging for backend systems.Refactor and enhance our Prometheus integration to facilitate configurable latency histogram buckets that can scale for high-traffic environments.Work collaboratively across teams on backend engineering priorities such as performance, reliability, and security.QualificationsRecent graduate with a Bachelor’s or Master’s degree in Computer Science or a related field.Proficient in Python and familiar with backend frameworks like FastAPI or Flask.Knowledge of logging best practices, error handling, and secure backend development principles.Exposure to monitoring and logging platforms such as Datadog, Splunk, Prometheus, or OpenTelemetry.Familiarity with database integration and troubleshooting (e.g., PostgreSQL, Redis).A strong drive to deliver high-quality backend code with attention to detail.

Join Our Team as a Senior Security EngineerAt LiteLLM, the leading AI Gateway trusted by industry giants such as Adobe, Netflix, and NASA, we empower developers with secure and reliable access to LLMs and associated services. We are seeking a talented Senior Security Engineer to establish robust security measures and observability tools as we scale our platform.Your Role:Become a cornerstone of our security team as our inaugural Security Engineer, tackling pivotal security challenges head-on.Key Responsibilities:Perform in-depth security assessments of the LiteLLM proxy codebase to uncover potential supply chain vulnerabilities.Develop and manage automated security scans for our Docker images, PyPI packages, and CI/CD workflows (including dependency scanning and secrets detection).Create and enforce secure-by-default configurations for both cloud and self-hosted environments (API authentication, IAM least privilege, key rotation).Implement and oversee intrusion detection systems and alerts tailored to model and API usage patterns.Lead incident response efforts and post-mortem analyses, including vulnerability assessments and stakeholder communication.Establish a formal CVE triage and disclosure protocol in collaboration with the engineering team.Conduct internal red teaming and adversarial testing to simulate real-world attacks and enhance our defenses.Collaborate with engineering teams to fortify release pipelines (signed builds, provenance checks, reproducible builds).Develop secure coding standards and conduct regular training sessions for developers focused on supply chain and dependency management.Maintain and update threat models as LiteLLM’s products and architecture evolve.

About CasaAt Casa, we believe that homes are among life’s most cherished assets, yet the experience of homeownership can often be frustrating. We are on a mission to revolutionize this experience by creating a next-generation personal property management solution tailored for single-family homes, ensuring that owning a home becomes a delight.Founded by early visionaries from Uber, CloudKitchens, and Levels, Casa is proud to be supported by a remarkable group of leading venture capital firms and angel investors. Our service is already live and delivering value to paying members.About the RoleWe are seeking a pioneering Senior Backend Engineer to join our dynamic team. In this role, you will be instrumental in designing, developing, and deploying robust backend systems that provide exceptional value to our members.You will collaborate closely with our founders, fellow engineers, and designers to craft innovative and impactful products. Your contributions will be vital in shaping our core systems and defining the culture of our founding team.Key ResponsibilitiesArchitect and develop scalable, secure, and reliable backend systems.Design and implement APIs that enhance our iOS applications and other client interfaces.Optimize the performance and scalability of our services to accommodate an expanding member base and increasing data volume.Stay updated with the latest advancements in backend technologies and cloud infrastructure, integrating new methodologies as suitable.Mentor a growing engineering team through code reviews and constructive feedback to uphold high-quality standards.About YouYou are passionate about building robust systems. You possess a deep understanding of backend architecture, API design, and database management, and are eager to construct systems that can scale with our increasing member base.You are excited to be a founding engineer at an early-stage startup, ready to contribute to the structure and systems being established, while also shaping the company culture.You thrive on building. You embrace the challenges and opportunities for growth that come with joining an early-stage company, and you excel in a fast-paced, evolving environment.You think and act decisively. Your proactive mindset will contribute significantly to our mission.

Join Point as a founding Senior Backend Engineer and be part of a team creating a modern and aspirational banking experience. We are dedicated to enhancing everyday debit cards with exciting rewards and benefits, setting a new standard in banking.Discover more about Point's innovative team and dynamic culture here.

About UsScribe is a dynamic workplace where talented individuals come together to achieve remarkable professional milestones. With over 94% of the Fortune 500 relying on Scribe to enhance their operations, our Workflow AI platform effortlessly captures and optimizes workflows, enabling teams to work smarter, faster, and with greater consistency.Since our inception in 2019, we have experienced rapid growth, amassing more than 5 million users across 600,000 businesses. Headquartered in San Francisco, we have been recognized as a LinkedIn Top Startup, valued at over $1 billion, and supported by prominent investors. Join us as we transform the way work is accomplished.Our Work CultureWe are passionate builders striving to perfect our crafts. We deeply value our colleagues and are committed to achieving success together. We adhere to the following principles:Accelerate ImpactRaise the BarEmpower Our UsersClarity is KindRapid LearningOne Team, One DreamRole OverviewWe are on the lookout for a Senior Backend Engineer to become a vital member of our Core Engineering Team, which drives some of the key functionalities at Scribe, such as document viewing and editing, content embedding and exporting, and supporting essential integrations. Millions of users depend on these workflows daily.In this position, you will take the lead on influential projects — from enhancing critical document features to developing new greenfield systems — while collaborating across the technology stack to deliver exceptional product experiences. As a backend engineer, you will deepen our systems, engage in technical discussions, and elevate the standard of engineering excellence within the team.Your ResponsibilitiesDevelop and deploy backend features within our Django monolith, covering core document workflows (viewing, editing, embeds, exports) as well as innovative greenfield projects that broaden Scribe's capabilities.Enhance performance and reliability by fine-tuning database queries, endpoints, and caching layers for critical user workflows.Collaborate with product managers, designers, and frontend engineers to create intuitive and dependable experiences at scale.Contribute to the architecture and design of our systems to improve overall efficiency and maintainability.

Join us on-site in the vibrant Dogpatch neighborhood of San Francisco as a Senior Backend Engineer.About GigaGiga, a pioneering voice AI startup, has successfully secured a $61M Series A funding and boasts an impressive roster of clients including DoorDash. We are dedicated to revolutionizing the customer experience through real-time AI agents capable of understanding emotions and resolving issues instantly, while being scalable across the globe's largest enterprises.This is a pivotal moment for our company as we aim to expand our horizons. Our ambition is to position Giga as the premier AI platform for enterprise automation, driven by our cutting-edge voice superintelligence. To realize this vision, we are seeking exceptional engineers to join our team.Our engineers are empowered to make significant impacts, influencing the daily experiences of millions. This unique opportunity allows you to work alongside visionary founders, contribute to a commercially successful venture, and help shape a generational company. Here are some highlights about us:Voice AI startup Giga raises $61M Series ADoorDash and Giga PartnershipGiga is trusted by leading B2C companies around the world to manage their most complex support and operational workflows across various communication channels including voice, chat, and email. If this excites you, we encourage you to apply!The RoleWe are searching for innovative backend engineers to develop the infrastructure and systems that power our AI agents. You will engage with a variety of technologies from data pipelines and analytics to real-time agent systems, delivering features that have a direct impact on our customers' operations.This position grants you the freedom to own challenges from start to finish—identifying the optimal approach, constructing solutions, and refining them based on your findings.What You'll Work OnHere are a few key projects from our current roadmap:Analytics overhaul: Transitioning our analytics service to ClickHouse and optimizing it for the next phase of growth.Hallucination detection: Enhancing our detection systems with the aim of minimizing hallucination occurrences.Browser agent utilization: Scaling and stabilizing our systems to ensure robust performance.

Join Revi as a Senior Backend Engineer and Shape the Future of Retail! At Revi, we're revolutionizing the retail landscape by merging the physical and digital worlds. As pioneers in the emergence of Web 3.0, we are transforming brick-and-mortar experiences into seamless digital interactions. With over three decades of evolution in web technology, we are on the brink of a new era where physical stores can harness the power of the internet to create unparalleled customer experiences.Our mission is to empower businesses with a state-of-the-art in-store digital platform that fosters lasting customer relationships. Our beautifully designed self-ordering system enables consumers to buy, pay, and earn rewards effortlessly. With fresh Series A funding of $14.5 million from top-tier Silicon Valley investors, we are poised to seize a multi-billion dollar market opportunity.Why Join Us Now? This is a pivotal moment for Revi. With our AI-driven in-store platform already live with growing restaurant chains, we are experiencing strong product-market fit and customer demand for new features. This is your chance to make a significant impact on our technical direction while building on a foundation of proven success.

About UsOur mission is to revolutionize language learning.Learning a new language can transform lives by providing access to diverse cultures, career opportunities, and communities. With two billion people globally striving to learn languages, the most effective method—one-on-one tutoring—remains difficult to access at scale and has not evolved significantly over the years. Speak is developing an AI-powered, human-like tutor that fits in your pocket, offering a conversation-first approach that allows learners to practice speaking, receive immediate feedback, and progress through thoughtfully designed lessons. This results in a comprehensive journey from novice to confident speaker across multiple languages.Since our launch in South Korea in 2019, Speak has emerged as the leading language learning app, now available to learners in various markets and over 15 languages. As one of the foremost AI companies globally, we have secured over $150 million in venture capital from notable investors such as OpenAI, Accel, Founders Fund, and Khosla Ventures, with a diverse team spread across San Francisco, Seoul, Tokyo, Taipei, and Ljubljana.About This RoleWe are seeking a Senior Backend Engineer, with full-stack capabilities, to help create innovative AI-driven product experiences. The ideal candidate should have experience collaborating closely with product, design, and machine learning teams, alongside a proven track record in developing polished consumer applications. This role is pivotal in shaping our latest products focused on programmatic, conversational language learning and will significantly impact the future direction of our company.Your ResponsibilitiesCrafting innovative AI-driven product experiences and user interfaces for the web.Full-stack engineering with a primary focus on designing scalable backend systems.Troubleshooting complex streaming challenges.Taking ownership of the complete development cycle of features from ideation to production.Writing code efficiently, shipping frequently, and making a substantial impact on our business by facilitating the scaling of our content.Your Profile5+ years of industry experience as a Backend Engineer, ideally within a fast-paced startup environment.Bonus: Experience on an app that has reached over 1 million users.Exceptional problem-solving and communication skills, along with the ability to work effectively in a collaborative environment.

About CasapCasap is an innovative Series A startup backed by over $30M in funding from renowned investors such as Emergence, Lightspeed, and Primary Ventures. Located in the vibrant tech hub of San Francisco, our team consists of experienced product leaders from industry giants like Robinhood and Chime. Our mission is to revolutionize banking operations by automating disputes and combating friendly fraud. Our solutions have garnered love and appreciation from everyday users and major financial institutions alike.We are currently seeking skilled Backend Engineers to join our team in San Francisco, eager to enhance the banking experience with our rapidly growing product that sits at the cutting edge of automation and Artificial Intelligence.

About UsAt Amari AI, we are transforming the logistics industry by replacing outdated manual workflows with intelligent AI agents. Our cutting-edge technology collaborates with human operators, automating complex, document-heavy processes to enable faster, more accurate shipment processing.Having surpassed the initial development phase, we have successfully achieved product-market fit and are experiencing exceptional growth, with over 100% month-over-month revenue increase. With $5 million raised from notable investors including First Round Capital and Pear VC, we are expanding our platform's capabilities. Our highly skilled team is composed of experts from leading tech companies such as Google, LinkedIn, and Salesforce, as well as prestigious academic institutions and AI research facilities.The RoleIn the capacity of Senior Backend Engineer at Amari AI, you will play a pivotal role in shaping the core data infrastructure that drives our AI-powered logistics platform. This position offers substantial ownership and autonomy, allowing you to design and scale systems responsible for ingesting, processing, and delivering real-time logistics data across various AI components, including pipelines, queues, databases, and search layers. Your technical leadership will significantly influence product reliability and performance from the outset.What You’ll DoDesign and scale distributed backend systems - including queue pipelines, service boundary definitions, and robust failure management strategies such as retry mechanisms, circuit breakers, and graceful degradation.Manage database design and reliability - focusing on schema design, indexing strategies, and query optimization tailored to specific use cases, along with data pipelines that maintain synchronization across systems at scale.Develop AI-driven search infrastructure for real-time logistics lookups and enhanced AI features.Provide technical leadership in a dynamic startup environment - establishing engineering standards, collaborating closely with the founders on strategic roadmaps, and mentoring future backend engineers.Who We're Looking For5–8 years of professional experience in backend software engineering, preferably in high-growth startups or large-scale production environments.Proven track record of end-to-end ownership of backend systems in fast-paced, ambiguous settings.Demonstrated capability in making informed architectural decisions under time constraints, with flexibility to adapt as requirements evolve.

About UsAt Conversion, we're revolutionizing marketing automation with our AI-native platform designed specifically for modern software companies. The landscape of marketing today is fragmented by outdated tools and disconnected workflows. We're here to change that.Our solution empowers growth teams to seamlessly manage their go-to-market strategies from acquisition to retention, all within a single, fast, and personalized interface powered by AI.With over $28 million raised from leading investors like Abstract Ventures, True Ventures, and HOF Capital, we are rapidly scaling with a current ARR of over $5 million and supporting more than 4,000 customers.Join our dynamic team of high achievers in San Francisco. If you're passionate about crafting innovative, product-led marketing solutions and want to collaborate with world-class teammates, we can't wait to meet you!Over $28M in funding$0 to $6M ARR in less than 12 months, serving over 4,000 customersElite team with backgrounds from Airbnb, Palantir, Pinterest, IMC Trading, Shopify, LinkedIn, Microsoft, and moreAbout the RoleHi! I'm James, co-founder and CTO of Conversion. I want to share what we envision for our ideal candidate to join our engineering team.The challenge we face at Conversion is significant. The absence of new entrants in the marketing automation space for decades speaks to the complexity of building such platforms. Our system must reliably manage gigabytes of data syncs per customer, orchestrate intricate workflows for mission-critical processes, and handle revenue-generating form submissions while delivering high-quality emails to millions of recipients.We're seeking a backend software engineer who is:Product-focused: We value engineers who can take ownership of features, design, and implement them from start to finish. The best product managers often come from engineering backgrounds, as they can effectively assess both the benefits and costs of new features.

About MeshAt Mesh, we are on a mission to revolutionize how consumers transact with any asset. Despite the existence of trillions of dollars in tokenized assets, they remain largely untapped for daily commerce. Mesh is closing this gap by transforming crypto payments into a reliable, practical, and widely accepted method of transaction. With a robust orchestration engine paired with an intuitive consumer app, we aim to unlock liquidity on a global scale. Supported by renowned investors including PayPal Ventures, Paradigm, and Galaxy Ventures, we are laying the groundwork for the future of the global economy. Join our journey!OverviewWe are on the lookout for a skilled and seasoned Backend Engineer to become an integral part of our team. In this role, you will be pivotal in the design, development, and maintenance of our cutting-edge software solutions. Mesh operates globally, with our headquarters located in San Francisco, California. The ideal candidate will collaborate closely with our Engineering and Product teams across North America, Europe, and India.

Join Us at ApiphanyAt Apiphany, we are at the forefront of foundational AI technology, driving innovation in physical product development. Our mission is to support global leaders in automotive, aerospace, medtech, and energy sectors by transforming complex, unstructured technical data into actionable insights in real-time. With investment from notable firms like Markforged, Databricks, GM, and Character, we aim to redefine engineering decision-making, simplifying complexity for top-tier manufacturers worldwide.Our cutting-edge models are designed to tackle the intricacies of engineering and manufacturing, incorporating a deep understanding of physics, design specifications, and program constraints. We are a compact yet highly skilled team composed of builders from prestigious institutions including Stanford, Berkeley, MIT, UW, and CMU, alongside industry veterans from GM, Ford, and Genesis Therapeutics. Together, we share a passion for revolutionizing hard-tech and establishing a company that sets the standard in our field.Your RoleIn the role of Senior / Staff Backend Engineer at Apiphany, you will take charge of designing, building, and scaling the essential systems that drive our intelligence platform. You will be responsible for developing low-latency backend services and APIs capable of processing extensive engineering data streams, maintaining security and compliance, and architecting systems for optimal performance and availability. Additionally, you will influence long-term architectural decisions, guide your colleagues through mentorship, and establish best practices for scaling backend services to deliver robust and high-performance applications.

Join our dynamic team at unify as a Senior Backend Software Engineer, where you will leverage your expertise to design and implement robust backend systems that enhance our product's performance and scalability. You will collaborate closely with cross-functional teams to develop innovative solutions that meet our customers' needs.

Complete is dedicated to transforming compensation into a strategic advantage for the world's most impactful organizations. We are modernizing compensation processes by moving away from traditional spreadsheets and embracing innovative solutions for leveling, compensation cycle planning, total rewards statements, and dynamic candidate offers, among other features!Why Choose Complete?We recognize that compensation plays a crucial role in a company's success and represents one of their largest expenditures. Unfortunately, many enterprises lack the know-how to implement a robust compensation strategy, which can lead to significant financial and operational inefficiencies. We believe that organizations with innovative compensation strategies are the ones thriving in today's market.Complete is proudly supported by Accel and Y Combinator. We have the privilege of being advised by industry leaders such as Will Larson (CTO of Carta), Eric Sagar (COO of Plaid), and Mathilde Collin (CEO of Front).About Us We are product-focused builders who prioritize quality, believing software in the 21st century should be intuitive and require minimal training. Our culture is characterized by low ego and rapid execution. We understand that success doesn't come from large teams, but rather from a focused approach. As a tight-knit team of fewer than 10 members, we combine exceptional product and business insights, enabling us to effectively compete and win against larger competitors. This is your opportunity to join us at the ground level!⭐ Our clientele ranges from startups like Linear and Brex to publicly traded companies such as Roblox, and we continually add remarkable brands to our roster. We work collaboratively in our San Francisco office, located near Embarcadero BART, 4-5 days a week. We believe that bringing talented individuals together in person accelerates our success as a startup.What We Are SeekingAt this pivotal moment in our growth, Complete is looking for experienced Senior or Staff Engineers to help us scale towards our ambitious revenue goals. If you thrive in a fast-paced environment and possess a strong sense of ownership, you may be a perfect fit. Our team is passionate about delivering exceptional customer experiences and eager to challenge industry norms. At Complete, you will focus on:Ownership: We value experienced professionals who take initiative and lead projects with confidence.Collaboration: Work closely with cross-functional teams to ideate and implement innovative solutions that drive results.Continuous Improvement: Stay ahead of technological advancements to enhance our product offerings and optimize user experiences.

Join Plaud as a Backend Engineer and be part of an innovative team driving the future of technology. You will design, develop, and maintain scalable backend systems that enhance our product offerings and improve user experiences. This is an exciting opportunity to work with cutting-edge technologies and collaborate with talented professionals in a fast-paced environment.

Be Part of a Transformative Team in HealthcareThe Issue: Patients like Sarah face devastating choices, delaying critical cancer treatments due to exorbitant monthly costs of $30,000. Similarly, Marcus's autoimmune condition deteriorated as he navigated insurance hurdles.Over half of essential prescriptions are abandoned due to:High Costs: Patients with hefty copays abandon treatments five times more frequently.Complex Insurance Systems: Prior authorization difficulties obstruct access to necessary medications.Logistical Issues: Managing medical visits for treatments can be overwhelming.Our Approach: At Neon, we leverage advanced technology to dismantle these barriers by automating patient access workflows for pharmaceutical companies—this includes automating authorizations, simplifying benefits verification, and providing financial assistance options.Our Vision: We aspire to create a world where every patient can effortlessly navigate the healthcare system, especially in critical times. Join us in ensuring access to medication is driven by medical necessity rather than bureaucratic hurdles or financial limitations. Our success translates to patients gaining access to life-saving treatments without financial ruin, while we develop a robust AI automation framework for healthcare's leading enterprises.Why Work with Us?Aspiring for Greatness: We’re not mere dreamers; we’re experienced builders on a mission to establish a $200B+ enterprise akin to Palantir or ServiceNow, catering to major healthcare organizations.Seasoned Founders: Our team is composed of successful entrepreneurs, Y Combinator and MIT alumni, alongside former engineers from Tesla and Google.Rapid Growth with Stability: In just four months, we transitioned from concept to securing seven-figure contracts in an industry where sales cycles typically span 12-18 months. We are profitable and laser-focused on execution.Exceptional Backing: Our funding comes from top-tier Silicon Valley VCs who have supported unicorns like DoorDash and Lyft, complemented by strategic healthcare investors with deep industry ties.Significant Impact & Opportunities: Collaborate at the nexus of agentic AI, healthcare innovation, and transformative patient outcomes.Accelerate Your Career: Join us early and grow swiftly as we strive for leadership in healthcare automation.

Join Kiddom as a Senior Backend Software EngineerAt Kiddom, we are revolutionizing the education landscape by providing a platform that champions equity and growth for all students. Our innovative approach integrates top-tier instructional materials with engaging digital learning tools, allowing schools and districts to take charge of their curriculum. With our exceptional curriculum management capabilities, we deliver personalized learning experiences that cater to the specific needs and aspirations of local communities. Our robust data insights empower educators and leaders to enhance instructional strategies, improve school programs, and foster professional development.We are seeking a dedicated Senior Backend Software Engineer to join our dynamic team, where you will play a crucial role in creating the next generation of scalable backend services. Your responsibilities will include the design, development, and maintenance of high-performance systems that underpin our applications.The ideal candidate will possess significant experience in backend technologies and system architecture, coupled with a genuine passion for addressing complex challenges faced by our users. Mentoring fellow engineers and sharing your expertise will be key aspects of this role. A strong affinity for education, an understanding of teachers' needs, exceptional product insight, and a commitment to Kiddom's mission and vision are essential.