Senior Staff Software Engineer & Data Scientist - Large Model Evaluation

Waymo is at the forefront of autonomous driving technology, striving to become the world’s most reliable driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has dedicated itself to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access and prevent countless lives lost to traffic accidents. Our cutting-edge Waymo Driver is the backbone of our fully autonomous ride-hailing service and can be implemented across various vehicle platforms and applications. Having completed over ten million rider-only trips, our technology has accumulated experience from autonomously driving over 100 million miles on public roads and tens of billions of miles in simulations across more than 15 U.S. states.The Large Model Evaluation team is pivotal to Waymo's AI mission. As advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs) continue, we are constructing state-of-the-art AI systems capable of addressing the full spectrum of real-world driving complexities. Central to our success is our capacity to measure progress effectively. Robust evaluation is crucial for deploying any large model, and at Waymo, the challenge is particularly intricate and safety-critical. We seek quantitatively-minded engineers to innovate and propose new methods for assessing the machine learning models integrated into the Waymo Driver.

Waymo is a pioneering force in autonomous driving technology, dedicated to becoming the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo focuses on developing the Waymo Driver—The World's Most Experienced Driver™—to enhance mobility access and significantly reduce traffic-related fatalities. The Waymo Driver powers our fully autonomous ride-hailing service and is adaptable across various vehicle platforms and applications. With over ten million rider-only trips completed and extensive experience from driving more than 100 million miles on public roads, complemented by simulations totaling tens of billions of miles across over 15 U.S. states, we are at the forefront of our industry.The Large Model Evaluation team plays a crucial role in advancing Waymo's AI ambition. As we leverage advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs), our goal is to create cutting-edge AI systems that navigate the complexities of real-world driving scenarios. Our progress hinges on our ability to measure performance effectively. With robust evaluation being essential for deploying large models, the challenges we face are particularly intricate and safety-critical. We are seeking engineers with a quantitative mindset to explore and establish innovative methods for assessing the machine learning models utilized in the Waymo Driver.

Waymo is at the forefront of autonomous driving technology, striving to become the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has been dedicated to developing the Waymo Driver—The World’s Most Experienced Driver™—with the goal of enhancing mobility access while aiming to reduce the tragic loss of lives due to traffic accidents. The Waymo Driver not only fuels our fully autonomous ride-hailing service but is also adaptable across various vehicle platforms and applications. With over ten million trips solely for riders and extensive experience from autonomously navigating over 100 million miles on public roads, alongside tens of billions of miles in simulation across more than 15 U.S. states, we are leading the charge in this transformative technology.The Large Model Evaluation team plays a pivotal role in advancing Waymo's AI vision. As we integrate cutting-edge Large Language Models (LLMs) and Vision-Language Models (VLMs), our aim is to construct sophisticated AI systems capable of addressing the multifaceted challenges of real-world driving. Central to our achievements is the ability to accurately measure our progress. In a landscape where robust evaluation serves as a critical barrier to deploying large models, the intricacies of this task at Waymo are particularly complex and safety-sensitive. We seek quantitatively driven engineers to innovate and propose novel methodologies for evaluating the ML models utilized in the Waymo Driver.

Waymo is at the forefront of autonomous driving technology with a vision to become the world's most reliable driver. Originating from the Google Self-Driving Car Project in 2009, Waymo is dedicated to developing the Waymo Driver—The World's Most Experienced Driver™—which aims to enhance mobility access while significantly reducing traffic-related fatalities. The Waymo Driver is the backbone of our fully autonomous ride-hail service and is adaptable across various vehicle platforms and applications. To date, the Waymo Driver has successfully conducted over ten million rider-only trips, leveraging its experience from over 100 million miles driven on public roads and billions of miles in simulation across more than 15 U.S. states.We are in search of passionate and skilled engineers to join our team and spearhead the post-training process for large language models (LLMs) in the realm of autonomous driving. You will collaborate with an exceptional team of researchers and engineers to innovate and elevate the next generation of advanced AI models.

Waymo is at the forefront of autonomous driving technology, dedicated to becoming the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, our focus has been on developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access and save countless lives lost to traffic accidents. The Waymo Driver underpins our fully autonomous ride-hail service, with applications across various vehicle platforms and use cases. With over ten million rider-only trips facilitated, our technology has autonomously navigated over 100 million miles on public roads and tens of billions of miles in simulation across more than 15 U.S. states.The Safety Evaluation team acts as the guardians of safety, integrating data analysis expertise and thorough validation into our autonomous driving systems. We work in close collaboration with engineering teams to ground the evaluation of machine learning-intensive systems in safety principles. Our deep knowledge permeates every aspect of our safety-critical systems, ensuring that every decision made by our vehicles is supported by rigorous analysis and sound reasoning. Our mission extends beyond mere assessment to innovation, as we develop advanced ML techniques specifically designed for the intricacies of collision evaluation. Ultimately, we are paving the way for safer roads and a future in which Waymo vehicles navigate the complexities of the real world with confidence.As a Senior Data Scientist specializing in Safety Evaluation at Waymo, you will be pivotal in ensuring the safety and reliability of our autonomous vehicles. Your role will encompass a wide range of data analysis, model development, and evaluation tasks. While no two days are the same at Waymo, your core responsibilities will include:Creating safety evaluation frameworks essential for enhancing ML models.Designing and executing comprehensive data analysis pipelines and validation frameworks.Utilizing statistical methods and ML models to analyze historical/simulation data and evaluate safety performance.Collaborating with cross-functional teams such as Simulation and Planning to guarantee thorough safety evaluations across the autonomous driving system.

Waymo is at the forefront of autonomous driving technology, committed to becoming the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has dedicated itself to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access while significantly reducing traffic-related fatalities. Our Waymo Driver powers a fully autonomous ride-hailing service and is adaptable to a diverse array of vehicle platforms and applications. With over ten million rider-only trips and experience from more than 100 million miles driven autonomously on public roads, Waymo is shaping the future of transportation.Our simulation environment is among the most sophisticated ever created, utilizing deterministic logic, physical dynamics, and cutting-edge Generative AI to establish a training ground for the Waymo Driver. The Simulator Evaluation team tackles the challenging question: How can we mathematically validate that a virtual world is perceptibly 'real'?We are on the lookout for a Staff Software Engineer who will take on the role of Technical Architect in this field. You will operate at the intersection of software engineering and AI, ensuring our simulated environments—whether governed by explicit rules or foundational models—accurately reflect reality.In this Staff-level position, you will report directly to a Senior Staff Software Engineering Manager and function as a Technical Lead, connecting intricate technical metrics with overarching product strategies.Your Responsibilities Will Include:Architecting Evaluation Standards: Define the 'Definition of Done' for simulation realism, anticipating product objectives (e.g., operational capabilities in snow or highway driving) and designing the evaluation roadmap to ensure our simulation fidelity evolves in alignment with onboard requirements.Acting as the System Critic: Create comprehensive mathematical frameworks to validate our hybrid virtual world, determining the balance between diverse evaluation needs—from verifying logical rules and dynamics to assessing the distribution quality of generative AI models.Building at Scale: Lead the development of large-scale, extensible evaluation platforms (in C++/Python), ensuring our metric pipelines are robust distributed systems capable of delivering clear, reproducible insights on petabytes of data.Providing Strategic, Cross-Functional Leadership: Serve as the technical liaison across various organizations, closely collaborating with AI research and other simulation teams. The evaluation workflows you design will facilitate rapid innovation and inform research trajectories.

Waymo is a pioneering company in autonomous driving technology, dedicated to becoming the world’s most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has established the Waymo Driver—The World’s Most Experienced Driver™—with a mission to enhance mobility access and save lives lost in traffic accidents. The Waymo Driver powers our fully autonomous ride-hailing service and can be integrated across various vehicle platforms and applications. Having completed over ten million rider-only trips, our technology has driven more than 100 million miles on public roads and tens of billions in simulations across over 15 U.S. states.The Perception team is responsible for developing systems that learn the spatial-temporal representations and semantic meanings of the environment surrounding our autonomous vehicles (AVs). We collaborate closely with downstream teams to optimize and integrate our work into the Waymo Driver, conduct research to solve real-world challenges, and work alongside research teams at Alphabet. With access to millions of miles of diverse driving data from various sensors, we empower engineers like you to (1) create methods for efficient continuous learning from extensive real-world data, (2) develop scalable models and training methodologies, (3) analyze real-world behaviors to create systems that can navigate complexities, and (4) optimize models for both onboard and offboard hardware.In this hybrid role, you will report to a Technical Lead Manager.

At Databricks, we are committed to empowering data teams to tackle some of the world's most significant challenges—ranging from revolutionizing transportation to accelerating medical breakthroughs. Our mission is realized through the development and management of the foremost data and AI infrastructure platform, enabling our clients to derive deep insights from their data to enhance business outcomes. Founded by engineers and driven by a passion for customer success, we eagerly embrace each opportunity to confront technical hurdles, from innovating next-generation UI/UX for data interaction to scaling our services across millions of virtual machines. Our journey has just begun.As a Senior Staff Software Engineer, you will be a pivotal member of the executive leadership team within the Data Platform organization. You will utilize cutting-edge AI developer tools and methodologies to construct the Data Intelligence Platform, which will facilitate automated decision-making across Databricks. This role involves collaboration with distinguished Tech Leads, product teams, data scientists, and others. You will shape the future of various tools, including logging, orchestration, data transformation, metric stores, governance platforms, and data consumption layers.

Waymo is a leader in autonomous driving technology dedicated to becoming the most trusted driver in the world. Originating from the Google Self-Driving Car Project in 2009, our focus has been on creating the Waymo Driver—an exceptional driver that enhances mobility access while significantly reducing traffic-related fatalities. The Waymo Driver not only supports our fully autonomous ride-hailing service but can also be adapted to various vehicle platforms and applications. With over ten million rider-only trips completed, our technology has safely navigated more than 100 million miles on public roads and processed tens of billions of miles in simulation across over 15 U.S. states.The Perception Data team at Waymo plays a critical role in shaping the strategy and technical direction for all data utilized in training and evaluating the Waymo Driver's perception capabilities. We manage the complete data lifecycle, developing automated systems and 'infrastructure-as-product' solutions that convert vast amounts of driving sensor data into high-quality training datasets. Our team tackles complex challenges, including active learning loops and open-vocabulary modeling, bridging raw data with advanced machine learning techniques.By integrating data ingestion, curation, and evaluation into a unified ecosystem, we facilitate the swift development of foundational models and next-generation perception systems. We work closely with Machine Learning, Infrastructure, and Evaluation teams to address intricate data challenges, ensuring our models can effectively interpret the long-tail of rare events. Ultimately, our contributions lay the groundwork for the Waymo Driver to operate safely in diverse environments.

Waymo is at the forefront of autonomous driving technology, dedicated to becoming the world’s most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has been committed to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance mobility access and save lives lost to traffic accidents. The Waymo Driver not only powers our fully autonomous ride-hailing service but can also be adapted to various vehicle platforms and applications. With over ten million rider-only trips under its belt, the Waymo Driver has driven autonomously over 100 million miles on public roads and completed tens of billions of miles in simulation across more than 15 U.S. states.The Planner Evaluation team tackles one of the primary challenges in autonomous driving: assessing and elevating the quality of the software steering the car. We seek data-driven software engineers and data scientists who are passionate about autonomous vehicles and adept at utilizing complex data to inform decision-making. If this excites you, we encourage you to apply!This position follows a hybrid work model and reports to an Engineering Manager.

About Our TeamThe GeminiApp team is dedicated to crafting a universal AI assistant that will transform how billions of individuals engage with technology. Our vision is to develop a personal and proactive life assistant that significantly boosts productivity and creativity, enabling users to achieve more in their daily lives. Join us as we redefine human-AI interaction at scale.About the RoleAs a Senior Data Scientist on the GeminiApp team, you will play a crucial role in shaping our product strategy while leading a dynamic team of data professionals. Your expertise in data will be pivotal in developing a uniquely proactive assistant, ensuring our strategic initiatives are firmly grounded in actionable insights. This is a high-impact position ideal for a data scientist eager to thrive in a fast-paced and innovative environment, passionate about creating user-focused experiences that will revolutionize our relationship with technology.Key ResponsibilitiesCollaborate with Product Management, Engineering, and UX leaders to formulate data-driven product strategies across various areas of GeminiApp.Translate complex, ambiguous questions into clearly defined problems, design robust experiments, and extract insights from large, complex datasets.Establish and implement innovative, goal-oriented metrics that drive product success.Develop and deploy statistical and machine learning models to understand user behavior, enhance product capabilities, and personalize the user experience.Effectively communicate findings and strategic recommendations to stakeholders, including executives.Promote a data-driven culture by integrating user engagement insights back into models.Work alongside the GenAI team to ensure model quality and feature adoption.Serve as a technical mentor for a global team, guiding junior team members in complex analyses and upholding best practices for high-quality, impactful work.

Join LinkedIn as a Senior Staff Applied Scientist specializing in Infrastructure Data Science. In this pivotal role, you will leverage your expertise to drive innovation and enhance the efficiency of our infrastructure systems. You will collaborate with cross-functional teams, developing cutting-edge data solutions that significantly impact our global user base.Your responsibilities will include conducting advanced data analyses, modeling complex systems, and implementing machine learning algorithms to optimize performance. You will also mentor junior scientists and contribute to a vibrant research culture, ensuring that LinkedIn remains at the forefront of technology and data science.

Waymo is at the forefront of autonomous driving technology, striving to become the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo has dedicated itself to creating the Waymo Driver—the world's most experienced driver™—with the aim of enhancing mobility access while preventing the countless lives lost in traffic accidents. The Waymo Driver powers our fully autonomous ride-hailing service and is adaptable across various vehicle platforms and use cases. With over ten million rider-only trips under its belt and extensive experience driving more than 100 million miles on public roads, complemented by extensive simulations across 15+ U.S. states, Waymo is leading the charge in safe and efficient transportation solutions.The Planner Evaluation team addresses a pivotal challenge in autonomous driving: enhancing and quantifying the software that drives our vehicles. We seek seasoned data-driven software engineers and data scientists passionate about autonomous vehicles and utilizing complex data to drive informed decisions. This is the perfect opportunity for someone eager to make a significant impact in the field!In this hybrid role, you will report directly to an Engineering Manager.You will:Develop performance metrics and driving quality signals for the Waymo driver utilizing various techniques, including statistics, mathematics, physics, algorithms, and machine learning.Innovatively use simulations and analyze real-world driving logs to assess driving performance.Design and implement robust methods to strengthen the correlation between changes in onboard software and simulated results.Advocate for code quality and best practices within a large and intricate code base.Analyze data and provide insights for enhancing metric quality and interpretability.Collaborate effectively with engineers, data scientists, statisticians, and leadership to deliver evaluation products and facilitate data-driven decisions.Rapidly validate the effectiveness of additional coverage, ensuring solutions are robust for customer teams to manage their own evaluations.

Waymo is at the forefront of autonomous driving technology, committed to becoming the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, we have dedicated ourselves to developing the Waymo Driver—The World’s Most Experienced Driver™. Our mission is to enhance mobility access while significantly reducing traffic-related fatalities. With over ten million rider-only trips powered by the Waymo Driver, we have accumulated extensive experience, having driven more than 100 million miles on public roads and tens of billions in simulation across 15+ U.S. states.The Waymo AI Foundations team is dedicated to creating machine learning solutions that tackle complex challenges in autonomous driving. Our goal is to ensure the safe operation of Waymo vehicles in various cities and under diverse driving conditions. We actively collaborate with other research teams within Alphabet, focusing on areas such as reinforcement learning, learning from demonstration, generative modeling, Bayesian inference, hierarchical learning, and robust evaluation.This position follows a hybrid work schedule and reports directly to a Staff Research Scientist / Tech Lead Manager.

Join Waymo, a pioneer in autonomous driving technology, dedicated to becoming the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, our mission is to enhance mobility access and eliminate traffic-related fatalities. The Waymo Driver has already facilitated over ten million rides, showcasing its capability after autonomously navigating over 100 million miles on public roads and trillions in simulations across more than 15 U.S. states.As part of the Release Evaluation team, you will play a key role in ensuring the safety of each Waymo Driver version prior to deployment. We develop automated systems to address rare and exceptional scenarios in autonomous driving, effectively identifying critical signals under constraints of time and resources. The Sampling and Efficiency team utilizes advanced importance sampling techniques and machine learning to enhance the statistical efficiency of our evaluation pipelines.

Job Title: Senior Staff Software Engineer - Enzyme Location: Mountain View, California At Databricks, we are driven by a passion for empowering data teams to tackle some of the world’s most intricate challenges—from revolutionizing transportation to accelerating medical advancements. We achieve this by developing and managing the leading data and AI infrastructure platform, enabling our customers to harness deep data insights to elevate their businesses. The Lakeflow Engineering team at Databricks oversees our comprehensive ETL product line, including innovative solutions like Materialized Views, Structured Streaming, and Delta Live Tables. We operate one of the largest data engineering platforms globally, processing exabytes of data daily for our extensive client base. We are in search of a committed technical leader to guide the Materialized Views engineering team. This team is tasked with creating next-generation Materialized View features that enhance both ETL workloads and query acceleration. In this role, you will engage in various areas to design and implement advanced systems that surpass current state-of-the-art: Incrementally maintaining materialized views Optimizing query performance Managing resources efficiently Developing effective storage structures Automating physical data optimization Your Impact: Address significant business needs at a large scale through your software engineering expertise. Deliver a robust, scalable, and fault-tolerant architecture for Materialized Views. Conduct low-level systems debugging, performance measurement, and optimization on large production clusters. Shape architecture design, influence product roadmaps, and assume ownership of new projects. Introduce tools that enhance automation and operational efficiency of services. Utilize your extensive experience to help preempt and resolve production issues. Plan and lead complex technical projects collaborating with multiple teams across the organization.

Waymo is at the forefront of autonomous driving technology with a commitment to becoming the most trusted driver in the world. Originating from the Google Self-Driving Car Project in 2009, Waymo is dedicated to developing the Waymo Driver—The World’s Most Experienced Driver™—which aims to enhance mobility access and save thousands of lives lost in traffic accidents. The Waymo Driver has been instrumental in powering Waymo’s fully autonomous ride-hailing service, accruing over ten million rider-only trips, and has autonomously navigated more than 100 million miles on public roads alongside tens of billions of miles in simulation across 15+ states in the U.S.We are launching an innovative team known as SCORPIO (SimEval Capacity Operations, Resource Planning, Infrastructure Optimization). This team will play a crucial role in optimizing the utilization of Waymo’s extensive simulation compute, storage, and network resources. SCORPIO will focus on creating data-driven models, metrics, and processes for demand forecasting, capacity planning, and resource allocation optimization, thereby enhancing developer experiences and maximizing infrastructure investment returns.

Waymo, the leader in autonomous driving technology, aims to be the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo is dedicated to developing the Waymo Driver—The World's Most Experienced Driver™—to enhance mobility access and significantly reduce traffic-related fatalities. With over ten million rider-only trips facilitated by our technology, which has autonomously navigated more than 100 million miles on public roads and countless simulations across more than 15 U.S. states, we are at the forefront of autonomous transportation.The Waymo Applied Research team is devoted to crafting machine learning solutions that tackle key challenges in autonomous driving, focusing on safely operating Waymo vehicles across diverse cities and under varying driving conditions. We thrive on collaboration, fostering partnerships with other research teams within Alphabet to push the boundaries of innovation.In this hybrid role, you will report to a Technical Lead Manager.

Be a Part of the Future of Home RoboticsAt sunday, we are pioneering the development of personal robots designed to liberate individuals from the burden of repetitive tasks. Our mission is to democratize access to generalized robotics, allowing families to reclaim precious time spent on mundane activities.After investing 18 months in assembling an exceptional team, securing investment, and validating our technology, we are on the lookout for passionate professionals eager to join us during this exciting phase of growth. If you are ready to channel your expertise into cutting-edge robotics innovation, we want to hear from you.Your RoleAs a Full Stack Engineer focused on ML Data & Evals, you will serve as the vital link between raw data and robotic intelligence. You’ll create the foundational infrastructure for our ML team to evaluate and deploy models effectively. Your contributions will expedite the research-to-production cycle, facilitating on-robot evaluations and visualizing model performance in intricate, real-world settings.Key ResponsibilitiesDesign Interactive Data Systems: Develop interfaces and engines that integrate robot and Skill Capture Gloves™ data, enabling efficient human-in-the-loop workflows, from episode annotation to seamless transitions between autonomous operation and manual teleoperation.Conduct Evaluations & Benchmarking: Create high-performance tools to compare model-driven motions with human-captured ground truths, aiding in the assessment of model advancements across various tasks.Data Processing & Orchestration: Design processing services that convert raw captures and sensor data into formats suitable for machine learning, ensuring a smooth transition from our global data collection systems to the models that power our robots.Adaptability in a Startup Environment: While your primary focus will be the ML Platform, our Full Stack Engineers should be versatile, readily adjusting to shifting priorities and contributing across different areas of the stack as we grow.

Waymo is leading the way in autonomous driving technology with a mission to be the world's most trusted driver. Originating from the Google Self-Driving Car Project in 2009, Waymo is dedicated to developing the Waymo Driver—The World’s Most Experienced Driver™—to enhance accessibility to transportation while preventing countless lives lost in traffic accidents. Our innovative Waymo Driver underpins our fully autonomous ride-hailing service and is adaptable to various vehicle platforms and applications. With over ten million rider-only trips facilitated by our technology, we have driven more than 100 million miles on public roads and logged tens of billions of miles in simulation across more than 15 U.S. states.The Perception Evaluation team at Waymo is pioneering advancements in autonomous driving, prioritizing the safety and reliability of our self-driving systems. Our team leverages state-of-the-art tools and methodologies to rigorously evaluate the performance of our perception systems, which are essential for the safe and effective operation of autonomous vehicles. We are currently seeking a Software Engineer to be instrumental in shaping the future of transportation and enhancing the quality and reliability of Waymo's autonomous vehicles.In this hybrid role, you will report directly to an Engineering Manager.