companyDatabricks logo

Senior Software Engineer - Distributed Data Systems

DatabricksMountain View, California
On-site Full-time $166K/yr - $225K/yr

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Unlock Your Potential

Generate Job-Optimized Resume

One Click And Our AI Optimizes Your Resume to Match The Job Description.

Is Your Resume Optimized For This Role?

Find Out If You're Highlighting The Right Skills And Fix What's Missing

Experience Level

Senior

Qualifications

Bachelor's degree (or higher) in Computer Science or a related technical field, or equivalent practical experience. Proficient in programming languages and technologies relevant to distributed systems and data processing. Experience in building scalable systems and a passion for solving complex technical challenges. Strong analytical and problem-solving skills, with a focus on performance optimization. Ability to collaborate effectively within a fast-paced team environment.

About the job

P-59

At Databricks, we are driven by a mission to empower data teams to tackle some of the most pressing challenges facing our world. Whether it's transforming transportation or accelerating medical innovations, we achieve this by creating and operating the foremost data and AI infrastructure platform. Our customers leverage deep data insights to enhance their business operations. Founded by engineers with a relentless commitment to customer satisfaction, we eagerly embrace every opportunity to address technical challenges—from designing cutting-edge UI/UX for data interaction to scaling our services across millions of virtual machines. And this is just the beginning.

In modern data analysis, advanced techniques such as machine learning surpass the capabilities of conventional SQL query engines. As a software engineer on the Runtime team at Databricks, you will be instrumental in developing next-gen distributed data storage and processing systems that not only outperform specialized SQL query engines in relational query performance but also offer the flexibility and programming abstractions necessary to support a diverse range of workloads, from ETL processes to data science applications.

Key projects you might work on include:

  • Apache Spark™: Contribute to the leading open-source framework for big data.
  • Data Plane Storage: Deliver reliable, high-performance services and client libraries for managing vast amounts of data on cloud storage solutions like AWS S3 and Azure Blob Store.
  • Delta Lake: A sophisticated storage management system that merges the scalability and cost-effectiveness of data lakes with the performance and reliability of data warehouses, offering features such as ACID transactions and time travel to simplify complex data engineering architectures.
  • Delta Pipelines: Simplify the management of extensive data engineering pipelines, allowing customers to effortlessly orchestrate, deploy, test, and upgrade thousands of data pipelines while minimizing operational burdens.
  • Performance Engineering: Develop the next-generation query optimizer and execution engine that is fast, tuning-free, scalable, and robust.

About Databricks

Databricks is a pioneering company at the forefront of data and AI technology, dedicated to helping organizations unlock the full potential of their data. Our innovative platform supports data teams in driving insights that lead to breakthroughs in various industries, from transportation to healthcare.

Similar jobs

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.