About the job
Join Databricks as a Staff Software Engineer and contribute to our innovative infrastructure that empowers data teams to excel with confidence and scalability. Our unified schema migration platform is the backbone of automated, observable, and developer-friendly workflows, ensuring that Databricks can swiftly evolve across thousands of databases and over 60 critical services.
Key Responsibilities:
- Schema Migration Platform: Design and maintain an automated infrastructure for schema changes in MySQL and TiDB, focusing on versioning, rollback, validation, and auditing to ensure swift, safe migrations.
- Scale, Reliability & Test Harness: Guarantee the platform’s operation across a vast array of global databases without downtime, while enhancing resilience, observability, and recovery measures, along with developing a test harness for migration and deployment validation.
- Developer Experience & CI/CD Integration: Create user-friendly tools and APIs for efficient schema and data change management, integrating migration workflows into CI/CD pipelines to boost speed and reduce friction.
- Schema & Data Management: Construct infrastructure for secure, large-scale schema and data operations, ensuring that backfill processes are monitored, retryable, minimally disruptive, and automated to maintain data integrity in line with schema evolution.
Your Impact:
- Enhance the scalability and reliability of schema and data operations across Databricks’ global presence.
- Facilitate predictable deployments by architecting robust test harnesses and automation tools.
- Improve the developer experience by delivering accessible and resilient tools that simplify complex migrations.
- Ensure the integrity of schema and data through safe and efficient backfill strategies.

