companyDatabricks logo

Senior Software Engineer - Database Engine Internals

DatabricksSan Francisco, California
On-site Internship $166K/yr - $225K/yr

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Unlock Your Potential

Generate Job-Optimized Resume

One Click And Our AI Optimizes Your Resume to Match The Job Description.

Is Your Resume Optimized For This Role?

Find Out If You're Highlighting The Right Skills And Fix What's Missing

Experience Level

Senior

Qualifications

What We Are Looking For: A deep passion for database systems, storage systems, distributed systems, language design, or performance optimization Experience in working towards a multi-year vision with incremental deliverables Strong motivation to deliver customer value and impact 5+ years of relevant experience in a similar field (preferred) Optional: PhD in databases or distributed systems

About the job

P-97

At Databricks, we are on a mission to fundamentally simplify the entire data lifecycle—from ingestion and ETL to BI and ultimately to ML/AI—through a unified platform. We envision a future where the traditional data warehouse architecture is transformed by an innovative architectural model known as the Lakehouse (CIDR 2021 paper). This open platform merges data warehousing with advanced analytics, effectively addressing critical challenges such as data staleness, reliability, total cost of ownership, data lock-in, and limited use-case support.

A key component in realizing this vision is the development of a next-generation decoupled query engine and structured storage system that surpasses the performance of specialized data warehouses while maintaining the flexibility of general-purpose systems like Spark™ to cater to a wide range of workloads, from ETL processes to data science applications.

As a vital member of our team, you will engage in the design and implementation of these next-generation systems that aim to leapfrog the current state-of-the-art in the following areas:

  • Query compilation and optimization
  • Distributed query execution and scheduling
  • Vectorized execution engine
  • Data security
  • Resource management
  • Transaction coordination
  • Efficient storage structures (encodings, indexes)
  • Automatic physical data optimization

About Databricks

Databricks is at the forefront of data innovation, providing a unified platform that integrates the entire data lifecycle. Our goal is to redefine how organizations manage and analyze data, leveraging the latest advancements in technology to enhance efficiency and performance.

Similar jobs

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.