PhD Research Intern - Multi-Modal Foundation Encoder for Perception

About the Internship Zoox is seeking a Software Engineering Intern focused on Perception Data at our Foster City, CA location. This role supports the development of software that improves how our autonomous vehicles understand and interpret their surroundings. What You'll Work On Collaborate with engineers to build and refine perception systems for autonomous vehicles Contribute to software projects that help vehicles navigate safely and efficiently Work alongside experienced professionals committed to advancing autonomous technology

Join Zoox as a Senior/Staff Software Engineer specializing in Perception & Sensing, where you'll work alongside a talented team of engineers from various domains including AI, robotics, and computer vision. This role offers the opportunity to innovate and implement cutting-edge technologies, contribute to research initiatives, and enhance key systems that drive our vision forward. Your expertise will play a pivotal role in the development of algorithms, software, and solutions that enhance our autonomous vehicle technology.

Join us in revolutionizing personal transportation at Zoox, where we're creating a safer, cleaner, and more enjoyable future. Our unique vehicle design is tailored for navigating dense urban environments with bidirectional driving capabilities and four-wheel steering, enabling seamless maneuverability in tight spaces without the need to reverse. We are seeking an experienced Engineering Manager to lead our Perception Occupancy and Rare Events team. In this role, you will oversee the development of Zoox’s cutting-edge 3D environment model utilizing multi-modal sensor data. Your expertise will guide the creation of innovative AI architectures that blend the latest academic advancements with our pioneering in-house innovations. Our algorithms are meticulously optimized to perform effectively on diverse real-world data and in simulations. As a leader, you will steer high-impact teams, charting technical roadmaps and achieving milestone goals. Collaboration with other AI domains—such as Simulation, Sensor and Hardware, and Systems Design and Mission Assurance—will be essential in delivering a top-tier object occupancy perception system that integrates modalities including vision, lidar, radar, and long-wave IR. You will mentor a diverse team, tackling some of the most intricate challenges in artificial intelligence, perception, and sensor fusion.

Join Zoox's innovative Perception Object Detection and Tracking team, where we focus on the sophisticated perception of dynamic objects and individuals. In this pivotal role, you will collaborate directly with machine learning model teams to integrate state-of-the-art models into our vehicle technology stack.You will have unparalleled access to premier sensor data and a robust infrastructure designed for rigorous testing and validation of your algorithms. We are pioneering new algorithms for segmentation, tracking, classification, and high-level scene comprehension, and you will have the opportunity to contribute to any (or all!) of these critical components.

Join Zoox as a Machine Learning Engineer Intern in Perception and gain invaluable hands-on experience with cutting-edge technology while working alongside industry-leading experts. Our internship program is designed for outstanding students who exhibit exceptional academic performance, extracurricular involvement, and a genuine passion for our mission at Zoox. As part of the Perception team, you will contribute to the development of systems that empower our autonomous vehicles to understand their surroundings. You will have the opportunity to work in one of several dynamic teams, focusing on multimodal models, scene understanding, vehicle recognition, and hazard detection, all while utilizing state-of-the-art machine learning techniques and real-world data.

Zoox’s Perception and Mapping Data team develops the systems behind high-definition maps for autonomous vehicles. These maps are essential for safe operation and expansion into new cities, as well as for keeping current routes accurate and up to date. Role overview This Software Engineer position centers on building and improving mapping systems that support Zoox’s autonomous driving technology. The work involves both creating new maps for additional locations and maintaining the quality of existing ones. What you will do Design and implement large-scale algorithms and workflows for HD mapping Build and maintain data pipelines to process mapping data efficiently Collaborate with cross-functional teams to address real-world challenges in autonomous driving Improve the speed and efficiency of mapping new areas and updating current maps Requirements Interest in solving challenging technical problems related to autonomous vehicles Experience with mapping systems, large data pipelines, or similar technical domains is valuable Location This role is based in Foster City, CA.

Join Zoox as a Senior Machine Learning Engineer specializing in Perception, where you will play a pivotal role in crafting and deploying state-of-the-art detection models that leverage integrated sensor data. This position focuses on pioneering projects tailored for the next generation of driving software, which includes developing joint detection and tracking models and innovating methods to transmit enriched embeddings to downstream models to enhance semantic understanding and uncertainty management.This unique opportunity allows you to significantly contribute to Zoox's venture into new horizons. By engineering more robust and adaptable models, you will aid in navigating complex and diverse scenarios, propelling our mission to achieve safe and dependable autonomous mobility. We seek an enthusiastic and driven individual who is eager to push the limits of perception technology in autonomous vehicles.

The Perception team at Zoox plays a pivotal role in advancing our autonomous vehicle technology by creating a sophisticated understanding of the world for our self-driving robots. Our expertise enables safe and efficient navigation through complex environments by leveraging advanced detection, classification, and tracking systems.As a Senior Software Engineer within the perception mapping team, you will be an integral player in Zoox’s online mapping initiatives. Your responsibilities will encompass designing, training, validating, and integrating machine learning models that accurately detect semantic elements within our environment. This role will engage you in all phases of ML development, including data collection, labeling, training, validation, and onboard integration. Your contributions will be crucial in achieving significant milestones in scaling and enhancing our autonomy capabilities, ultimately driving the success of Zoox.

At Zoox, we are pioneering a revolution in personal transportation by crafting a future that is safer, cleaner, and more enjoyable for everyone. Our innovative vehicle is uniquely designed for navigating the complexities of dense urban environments. As the only vehicle on the market featuring bidirectional driving capabilities and four-wheel steering, we excel at maneuvering through tight spaces and changing directions seamlessly, eliminating the need for reversing. Join us in shaping the future of mobility!We are currently expanding our dynamic software engineering leadership team and seeking a visionary Director of Perception. In this pivotal role, you will oversee the development of Zoox’s state-of-the-art 3D environment model, utilizing multi-modal sensor data. Our team thrives on innovation, combining cutting-edge AI architectures with the latest academic advancements and our proprietary solutions. Your expertise will drive the optimization of our algorithms to ensure peak performance across a diverse range of real-world scenarios and simulations.As the Director of Perception, you will lead high-impact teams, guiding their technical roadmaps and setting milestone objectives. Collaboration will be key as you work alongside other AI and software leaders, simulation experts, and our systems design and mission assurance teams. Your mission will include delivering an exceptional dynamic object perception system that integrates various modalities such as vision, lidar, radar, and long-wave IR across complex pipelines including segmentation, detection, classification, fusion, and tracking. You'll manage a diverse and skilled team, tackling some of the most challenging problems in artificial intelligence, perception, and sensor fusion.

Join the visionary Perception team at Zoox, where we are revolutionizing autonomous vehicle technology by creating an intricate understanding of our surroundings for self-driving robots. Our mission is to facilitate safe and efficient navigation within complex environments through advanced detection, classification, and tracking systems.As a pivotal software engineer on the perception mapping team, you will drive the development of Zoox's online mapping initiative. Your responsibilities will include designing, training, validating, and seamlessly integrating machine learning models that identify semantic map elements in real-world settings. You will engage in all facets of machine learning development, from data collection and annotation to training, validation, and onboard integration. Your contributions will be instrumental in achieving key milestones that enhance our scaling and autonomy capabilities, playing a crucial role in Zoox's success.

Onsite in Foster City, CA | at least 3 days onsiteThe Perception team seeks a talented Machine Learning Engineer to innovate advanced Computer Vision modules, significantly enhancing the onboard perception capabilities of our robotic fleet. This position will directly influence safety and operational efficiency. The ideal candidate will engage in the complete design and development process, encompassing data collection, dataset creation, and the design and implementation of machine learning models. Key Responsibilities:Architect and develop in-cabin Computer Vision ML modules for critical safety and monitoring applications.Implement advanced Perception model architectures and sophisticated training methodologies.Create high-quality datasets utilizing inputs from our sensor stack and large-scale data.Validate and optimize solutions through real-world driving scenarios, directly enhancing the safety and reliability of our autonomous systems. Essential Qualifications:Bachelor's Degree in Computer Science or Computer Engineering with a minimum of 4 years of relevant experience.Proficient in modern Computer Vision and Machine Learning, especially in areas like 3D reconstruction, detection, classification, and semantic segmentation.Experience in training and deploying Deep Learning models.Strong programming skills in C++ and/or Python.Exceptional communication abilities. Preferred Qualifications:Familiarity with any of the following:Experience in the self-driving industry.Expertise in integrating ML models within latency-sensitive systems.Proficiency in CUDA programming. Employee Benefits:Comprehensive Health Care Plan (Medical, Dental, and Vision).Life Insurance (Basic and Voluntary, including AD&D).Generous Paid Time Off (Vacation, Sick Leave, and Public Holidays).Opportunities for Training and Development.Retirement Plan options (401k, IRA).

Join the innovative Perception team at Zoox, where we harness the power of Generative AI to produce synthetic data that revolutionizes the training and evaluation of our autonomous systems. As a Machine Learning Engineer specializing in Generative AI, you will be at the cutting edge of developing and training advanced models for sensor-level scenario generation. By utilizing world models and radiance field techniques alongside our extensive proprietary datasets, your work will have a direct impact on enhancing the productivity, safety, and capabilities of Zoox's autonomous systems through real-world algorithm validation.

High-definition maps are essential for ensuring safe and scalable autonomous driving. We are on the lookout for driven engineers who are enthusiastic about tackling intricate, real-world challenges while developing cutting-edge mapping systems.This position involves collaborating with a diverse and cross-functional team to design and implement extensive HD mapping algorithms, workflows, and data pipelines. Your contributions will significantly enhance our capability to efficiently map new urban areas and continuously update existing maps on a large scale, playing a pivotal role in expediting our autonomous vehicle deployment.

Join us as an Autonomous Vehicle (AV) Safety Data Engineer, where you will be instrumental in creating and maintaining a formidable data infrastructure that highlights safety-relevant interactions. This role is pivotal in assessing AV events and steering the development of autonomy through insightful data analysis. You will design scalable data pipelines and metrics that capture reliable data signals, directly impacting our evaluation of system readiness and expediting safe deployments. Moreover, you will support vital tasks related to event triage and analysis.Key Responsibilities:Develop and enhance data miners to detect significant events within fleet and simulation datasets.Utilize advanced statistical and machine learning techniques to refine existing miners for improved accuracy and efficiency, ensuring high-quality signals.Create scalable data pipelines and dashboards to monitor miner performance and key safety metrics.Perform exploratory data analyses to identify trends, failure modes, and enhancements to vehicle behavior focused on critical safety incidents and system anomalies.Evaluate AV events concerning system behavior and hardware anomalies, considering safety, legal, and engineering principles.Establish and maintain clear triage guidelines for both onshore and offshore teams.Assist in regulatory reporting by providing relevant data and analyses.

About Zoox Zoox designs and builds fully autonomous electric vehicles, aiming for a future where transportation is safer, more sustainable, and accessible. The team values innovation, collaboration, and a forward-thinking approach to mobility. Internship Program Overview Zoox’s internship program offers hands-on experience with advanced technology and mentorship from experienced professionals in the field. The program looks for PhD interns with strong academic records, curiosity, and a commitment to Zoox’s mission. Engagement in activities beyond coursework is also valued. About the Physical AI Perception Internship This research internship sits within the Perception Semantics team, which focuses on developing AI systems that help robots interpret and interact with the physical world. Interns will explore topics in vision-language-action (VLA) modeling, world modeling, spatial reasoning, and mapping. The work spans both theoretical research and practical implementation. Projects are exploratory and research-driven, encouraging interns to investigate new ideas, create solutions, and test them in real-world scenarios. This role suits PhD candidates interested in pushing the boundaries of computer vision and embodied AI, with the opportunity to see their research have real impact.

Join our innovative Motion Planning team, where you will play a critical role in enhancing the driving behaviors of our robots as they navigate public roads. Your mission will be to determine the robot's destination and devise safe routes in complex environments, such as the bustling streets of Downtown San Francisco or the iconic Las Vegas Strip.This is an exhilarating challenge that demands close collaboration with a diverse group of specialists, including experts in Planner, Perception, Prediction, and Operations. Together, you will drive the advancement of cutting-edge algorithms that enhance our robot's capabilities in navigating intricate traffic scenarios.Safety is our top priority. You will employ a systematic approach by developing modifications to the Planner, conducting simulations and closed-course tests, and collaborating with our safety drivers for real-world testing.Key Responsibilities:Develop, debug, and test features and enhancements related to driving behaviors in our production codebase.Collaborate closely with senior engineers to interpret design specifications and produce robust, high-quality code.Utilize our comprehensive backend tools to execute regression tests and validate your modifications across various driving scenarios.Engage in on-vehicle drive reviews, assisting in data collection and generating actionable insights on driving performance.

Join Vertex Sigma Software as a Business Intelligence Engineer (BIE) and collaborate closely with our System Design and Mission Assurance Team. In this pivotal role, you will drive the team and leadership with actionable metrics and dynamic visualizations, ensuring we maintain the utmost standards of safety and operational excellence as we approach commercial deployment.Your collaboration with data scientists, data engineers, and a variety of cross-functional stakeholders will be essential in designing and implementing BI solutions that deliver crucial insights into data necessary for safety clearance and milestone tracking.Key Responsibilities:Collaborate with both technical and non-technical stakeholders to define requirements and deliver automated, actionable data solutions.Design, construct, and sustain scalable data models and ETL pipelines to guarantee high-quality, reliable data assets.Work closely with data engineers and data scientists to establish consistent and accurate metrics and data models.Create and manage impactful visualizations and dashboards that support data-driven decision-making.Enhance data literacy across the organization through the development of self-service tools and user training.Promote best practices in reporting and analytics, focusing on data integrity, scalability, validation, and comprehensive documentation.Transform business requirements into dependable data assets and provide analytics support under tight deadlines.

Join Sigma Software as a Database Engineer, where you will play a pivotal role in designing, building, and managing scalable and secure data storage solutions that underpin our applications and analytical processes. Collaborate with both application and data teams to craft schemas, enhance performance, and automate operations using AWS and infrastructure-as-code practices. Your contributions will significantly boost reliability, cut costs, and expedite delivery from development to production.Key Responsibilities:● Design and implement efficient and scalable database solutions tailored to business needs.● Install, configure, and upgrade database software and associated tools.● Monitor database performance while optimizing queries, indexes, and schemas for enhanced reliability and speed.● Establish and uphold database security measures to safeguard sensitive information.● Work alongside application developers to refine schema designs and SQL queries.● Diagnose and resolve incidents promptly across various environments.● Create and maintain comprehensive documentation for architecture, configuration, and operational procedures.● Engage in capacity planning and advocate for infrastructure enhancements.● Manage and support AWS database infrastructure, including RDS, Aurora, and Redshift.● Remain updated on emerging database technologies, methodologies, and industry best practices.

We are seeking a dynamic and detail-oriented QA Release Engineer to join our innovative team at Vertex Sigma Software. In this pivotal role, you will be responsible for managing the machine learning (ML) model merge queue and facilitating the release of new ML models for our vehicle systems. Your expertise will be instrumental in optimizing developer workflows and minimizing delays in branch updates.