companyDatabricks logo

Staff Software Engineer - Database Engine Internals

DatabricksBelgrade, Serbia
On-site Full-time

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Unlock Your Potential

Generate Job-Optimized Resume

One Click And Our AI Optimizes Your Resume to Match The Job Description.

Is Your Resume Optimized For This Role?

Find Out If You're Highlighting The Right Skills And Fix What's Missing

Experience Level

Mid to Senior

Qualifications

Your Responsibilities:Clarify requirements and guide design decisions for complex problemsCreate technical design documents and project plansDevelop innovative new featuresMentor junior engineersConduct testing and oversee production rollout, including monitoring. What We Are Looking For:A deep passion for database systems, storage systems, distributed systems, language design, or performance optimizationA comfort level in working towards a multi-year vision with incremental deliverablesA customer-centric approach with a focus on impact7+ years of experience in a related fieldOptional: PhD in databases or distributed systems

About the job

Join Databricks as we embark on a transformative journey to revolutionize the data lifecycle, from ingestion through ETL, BI, and into the realms of ML/AI, all within a unified platform. Our vision is to transition from traditional data warehouse architectures to the innovative Lakehouse paradigm, as detailed in the CIDR 2021 paper. This new architecture addresses critical challenges such as data staleness, reliability, total cost of ownership, data lock-in, and limited use-case support.

At Databricks, we are developing the next generation of decoupled query engines and structured storage systems designed to surpass specialized data warehouses in relational query performance. Our goal is to maintain the expressiveness and robustness of general-purpose systems, like Apache Spark™, to accommodate diverse workloads, ranging from ETL to advanced data science applications. You will play an essential role in this multi-year endeavor.

As a valued member of our team, you will be tasked with designing cutting-edge systems that leapfrog current state-of-the-art technologies in the following areas:

  • Query compilation and optimization
  • Distributed query execution and scheduling
  • Vectorized execution engine
  • Data security
  • Resource management
  • Transaction coordination
  • Efficient storage structures (encodings, indexes)
  • Automatic physical data optimization

About Databricks

Databricks is a leader in data and AI solutions, serving over 10,000 organizations globally. Our mission is to simplify the complexities of data management and analytics, making it accessible and efficient for everyone. We are committed to innovation and excellence in everything we do.

Similar jobs

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.