About the job
Mission Summary
The AI Data Foundry serves as the critical bridge linking our extensive data resources with the engineering teams that depend on them. We design and manage the essential services and infrastructure required to convert petabytes of raw data monthly into actionable insights for our Machine Learning and Autonomy engineers. This position is pivotal in driving our core initiatives:
- Data Processing: We are developing cutting-edge infrastructure to accommodate the demands of our rapidly expanding fleet and advanced ML engineering needs. Your responsibility will be to ensure that our core data pipelines are robustly engineered—performant, user-friendly, and highly fault-tolerant.
- Data Discovery: We are creating a platform that integrates data cataloging for search capabilities across our metadata stores, vector datastores, and various data warehouses to enhance our ML pipelines. You will collaborate with ML and autonomy stakeholders to develop this new discovery platform.
- Data Lineage: We are initiating the construction of our data lineage system to track data usage at Motional. You will partner with our stakeholders to ensure the widespread adoption of our system and address user pain points.
What You'll Be Doing
- Playing a key role in designing and building the next generation of data infrastructure.
- Building scalable backend services for data discovery and data lineage systems.
- Writing high-quality, maintainable code to process petabytes of data.
- Leading and advocating for high-level system design and conducting quality code reviews and technical contributions.
- Designing, building, and maintaining scalable data processing and access using cloud ETL technologies.

