companyDatabricks logo

Senior Software Engineer - Database Engine Internals

DatabricksMountain View, California
On-site Full-time $166K/yr - $225K/yr

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Unlock Your Potential

Generate Job-Optimized Resume

One Click And Our AI Optimizes Your Resume to Match The Job Description.

Is Your Resume Optimized For This Role?

Find Out If You're Highlighting The Right Skills And Fix What's Missing

Experience Level

Senior

Qualifications

What We Look For:A strong passion for database systems, storage systems, distributed systems, language design, or performance optimization. Proven experience working towards a multi-year vision with incremental milestones. Dedication to delivering customer value and making a meaningful impact.5+ years of relevant experience in a related field (preferred). Optional: PhD in databases or distributed systems.

About the job

At Databricks, we strive to revolutionize the data lifecycle from ingestion to ETL, business intelligence (BI), and machine learning (ML) with our unified platform. We envision a future where the traditional data warehouse architecture is superseded by an innovative architectural model known as the Lakehouse (CIDR 2021 paper). This cutting-edge approach integrates data warehousing with advanced analytics, effectively addressing significant challenges such as data staleness, reliability, cost of ownership, data lock-in, and limited use-case support.

A pivotal component of achieving this vision is the development of the next-gen (decoupled) query engine and structured storage system designed to surpass the performance of conventional data warehouses for relational queries while maintaining the flexibility of general-purpose systems like Apache Spark™. This will empower a wide range of workloads, from ETL processes to data science applications.

As a member of our team, you will engage in one or more of the following areas to design and implement these advanced systems that set new benchmarks:

  • Query compilation and optimization
  • Distributed query execution and scheduling
  • Vectorized execution engine
  • Data security
  • Resource management
  • Transaction coordination
  • Efficient storage structures (encodings, indexes)
  • Automatic physical data optimization

About Databricks

Databricks is on a mission to radically simplify the data lifecycle with a unified platform that integrates ETL, BI, and ML/AI processes. Join us in transforming the way organizations manage and analyze their data.

Similar jobs

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.