Senior Data Engineer - Data Engineering

The OpportunityJoin Enigma at a pivotal moment as we prepare to unveil our next generation of small business intelligence products, informed by years of industry experience. We are in search of a seasoned Senior Data Engineer who will play a crucial role in shaping the future of these innovative products. Your contributions will significantly enhance the accuracy, scalability, and predictive analytics of our data platform, directly impacting our clients' key business strategies.The RoleAs a Senior Data Engineer, your primary responsibility will be to design, construct, and manage our foundational small business data product. You'll work closely with cross-functional teams to develop new product functionalities, starting from raw data sources, through scalable data pipelines and models, to the customer interface. Our culture emphasizes customer focus and complete ownership, measuring your success by the value delivered and the technical excellence of our product.What You’ll DoIdentify and address the unmet needs of our customers through thorough data exploration, prototyping, and evaluation, culminating in the development of robust and maintainable data pipelines.Construct scalable and maintainable data systems that narrate the stories of millions of U.S. small businesses, utilizing tools like Databricks SQL, PySpark, and MLFlow.Leverage our pioneering data infrastructure designed for iterative development and experimentation, utilizing Dagster for orchestration and LakeFS for data version control.Employ a data-driven, first-principles approach to problem-solving, favoring straightforward and elegant solutions while being capable of deploying advanced models and techniques when necessary.Motivate your colleagues to excel while nurturing a collaborative and supportive team environment.What Makes This Role Exciting?Ownership and Impact: Collaborate directly with customers and internal teams to develop product features in a high-trust environment.Technical Challenge: Solve complex data and engineering challenges, balancing innovation speed with scalability and reliability.Learning: Engage with unique, impactful data challenges that demand creative solutions and cutting-edge methodologies.

Front Careers seeks a Senior Data Engineer based in San Francisco, CA. This position centers on building and maintaining data systems that power analytics and reporting throughout the organization. Key responsibilities Design and maintain data architecture to meet analytics and reporting requirements Develop, optimize, and support data pipelines for reliability and scalability Collaborate with teams across the company to improve data quality, accessibility, and security Enhance and maintain the data infrastructure supporting company operations Role overview This role focuses on ensuring that data flows efficiently and securely, enabling teams to access the information they need for business decisions. The Senior Data Engineer will play a central part in shaping how data is managed and used at Front Careers.

About AlembicAlembic is at the forefront of transforming marketing strategies, demonstrating the actual ROI of marketing initiatives. Our cutting-edge Alembic Marketing Intelligence Platform employs advanced algorithms and AI models to address this longstanding challenge effectively. By joining our team, you'll contribute to the development of tools that deliver unparalleled insights into how marketing influences revenue, empowering a growing roster of Fortune 500 companies to make data-driven decisions with confidence.About the RoleIn your role as a Senior Data Engineer at Alembic, you will play a crucial role in our data platform. You will be responsible for creating scalable and dependable data pipelines, optimizing storage solutions, and facilitating both real-time and batch analytics. Collaborating closely with data scientists, software engineers, and product leaders, you will design and implement robust data architectures that propel our mission forward.Key ResponsibilitiesDesign, develop, and maintain scalable ETL pipelines that efficiently ingest, process, and transform extensive volumes of structured and unstructured data.Optimize data storage solutions utilizing modern data lakehouse architectures and industry best practices to enhance cost-effectiveness, performance, and reliability.Collaborate with data scientists and engineers to seamlessly integrate machine learning models and analytical workloads into production environments.Ensure the integrity, quality, and security of data by implementing monitoring, alerting, and governance best practices.Work with cloud-based data warehouses and distributed data processing frameworks to support our data initiatives.Continuously assess and implement innovative technologies to enhance data infrastructure and operational efficiency.What We’re Looking For10+ years of experience in data engineering, software engineering, or a related field.Strong proficiency in SQL and Python for data processing.Experience with contemporary data warehousing and lakehouse solutions (e.g., Iceberg or similar).Expertise in distributed systems and big data technologies (Apache Spark, Hadoop, Kafka, Flink).Hands-on experience with cloud platforms (AWS, GCP, Azure) and related data services.Deep understanding of data management and governance practices.

About tvScientific tvScientific is the pioneering CTV advertising platform specifically designed for performance marketers. We harness extensive data and advanced technologies to automate and enhance TV advertising, ultimately driving measurable business results. Our platform seamlessly integrates media buying, optimization, measurement, and attribution into a single, efficient solution. Developed by industry veterans with deep expertise in programmatic advertising, digital media, and ad verification, our CTV performance platform offers advertisers a reliable avenue to expand their business. As a Senior Data Engineer at tvScientific, you will play a crucial role in establishing the robust data infrastructure that supports our data-intensive operations. You will work in collaboration with cross-functional teams to refine our core data pipelines, ensuring efficient scaling as we grow, and optimizing data storage solutions. This individual contributor role requires you to define and execute a strategic vision for data engineering within the company. Key Responsibilities: Develop and implement a robust data infrastructure in AWS, utilizing Spark with Scala. Enhance our core data pipelines to efficiently accommodate our significant growth. Optimize data storage solutions in appropriate engines and formats. Collaborate with cross-functional teams to design data solutions that align with business objectives. Construct fault-tolerant batch and streaming data pipelines. Leverage and optimize AWS resources while scaling design. Work closely with Data Science and Product teams to achieve collective goals. Success Metrics: Successful establishment of scalable and efficient data infrastructure. Timely delivery and optimization of data assets and APIs. High attention to detail in the implementation of automated data quality checks. Effective collaboration with cross-functional teams. What We’re Looking For: Proven experience in production data engineering. Expertise in Spark and Scala, with a track record of building data infrastructure using these technologies. Familiarity with data lakes, cloud warehouses, and various storage formats. Strong proficiency in AWS services. Advanced SQL skills for data manipulation and extraction. Exceptional written and verbal communication abilities. Bachelor's degree in Computer Science or a related discipline. Nice to have: Experience with additional data processing frameworks.

About Formation BioFormation Bio is a cutting-edge pharmaceutical company harnessing the power of technology and AI to transform drug development. As advancements in AI and drug discovery generate more candidate drugs than the industry can efficiently progress through clinical trials, Formation Bio, originally founded as TrialSpark Inc. in 2016, has developed innovative technology platforms and processes to streamline every aspect of drug development and clinical trials. Our mission is to alleviate the bottleneck in drug development to ensure that new medicines reach patients more swiftly. We collaborate, acquire, or in-license drugs from pharmaceutical companies, biotechs, and research organizations to advance programs beyond clinical proof of concept, ultimately delivering new therapies to patients. Our esteemed investors include a16z, Sequoia, Sanofi, Thrive Capital, Sam Altman, John Doerr, Spark Capital, SV Angel Growth, and many others. Explore more about us through the following links:Our Vision for AI in PharmaOur Current Drug PortfolioOur Technology & PlatformAt Formation Bio, our core values empower our mission to revolutionize the pharmaceutical landscape. Every member of our team embodies these values and contributes significantly to our goal of expediting access to new treatments for patients. About the PositionWe are seeking a Senior Data Engineer to become a vital part of our Scientific Data Intelligence (SDI) team at Formation Bio. Your primary responsibility will be to enhance and scale the data infrastructure that supports our drug development platform. This role will require you to collaborate closely with the Data Science team, developing coherent, well-structured data models, feature engineering pipelines, and evidence layers that are foundational to their scientific endeavors. In addition, you will be tasked with ingesting data from diverse sources—APIs, files, databases, and licensed third-party datasets—and transforming it into clean, analytics-ready assets utilizing modern data tools.This position is ideal for an individual who takes pride in building robust systems: reliable data pipelines, well-organized data models, and frameworks that colleagues can confidently rely upon. You will work in conjunction with Data Scientists and other engineering experts to propel our mission forward.

About UsAt Rox, we are dedicated to empowering individuals to achieve their greatest potential. Our innovative platform enhances sales efforts through autonomous revenue agents, allowing sellers to prioritize their expertise in selling. Just as coding agents transformed engineering, revenue agents amplify customer interactions.We are revolutionizing the revenue stack by developing the world’s first revenue operating system, encompassing everything from the application layer to systems of context. At Rox, we envision a future where humans evolve into orchestrators while agents handle the complete customer lifecycle.Our solutions support Global 2000 leaders across sectors such as banking, construction, and AI, partnering with industry giants like Ramp and Cognition.Our success stems from a united belief in our mission and an unwavering commitment to making it a reality.The TeamOur world-class team is the backbone of our innovative approach to redefining business operations.Our team members have:Founded and successfully exited companiesHeld top roles at Google, AWS, Confluent, and New RelicWon gold medals in international mathematics competitionsPublished groundbreaking research papersWe are proud to be backed by leading investors, having raised $50 million from Sequoia (Alfred Lin), General Catalyst (Hemant Taneja), Google Ventures, Elad Gil, and Chris Ré.Core PrinciplesTaste: Craft beautiful experiences.We meticulously focus on every detail, striving to ensure that each interaction not only helps sellers accomplish their tasks but also enhances their experience. We are relentless in our pursuit of excellence, always exploring new ways to delight our sellers.Obsession: Commit unreasonably.We are dedicated to our craft, responding to customer needs proactively and driving value even before they ask. Our commitment to continuous learning and self-improvement is unwavering.Action: Get it done.Execution is key; we prioritize thoughtful yet swift decision-making and immediate delivery. Trust is essential in our field, and we earn it through our actions.

Join Unify as a Senior Data Engineer, EnrichmentAt Unify, we are pioneering the first AI-driven action system for revenue teams, enabling organizations to transform their outbound efforts into a powerful growth engine. Our technology makes go-to-market strategies observable, repeatable, and scalable. Established in 2023 by visionaries from Ramp and Scale AI, our team boasts experience from industry giants such as Airbnb, Meta, Waymo, and Perplexity.In 2024, we achieved an impressive 8x revenue increase and cater to esteemed clients like Perplexity, Cursor, SoFi, and Justworks. We are a dynamic and high-energy team that has successfully raised $58M from prominent investors including Thrive, Emergence, and OpenAI. Help us shape the future of go-to-market strategies - come aboard!Role OverviewAs a Senior Data Engineer focusing on Enrichment, you will take ownership of the systems that create and uphold the most extensive and high-quality first-party contact dataset available. Your responsibilities will include designing data pipelines that assimilate information from various vendors, developing intelligence to evaluate data source reliability, and establishing a quality framework that secures Unify's data as a competitive advantage.

About WorkWhileAt WorkWhile, we're driven by a mission to empower hourly workers to achieve a more stable and fulfilling life. With over 80 million Americans engaged in hourly employment, many face unpredictable schedules, minimal wages, and limited benefits, resulting in financial uncertainty.Our innovative approach aims to transform frontline work by fostering a flexible and efficient labor market. Leveraging AI technology, the WorkWhile platform connects businesses with exceptional talent swiftly, while providing workers with same-day pay, complimentary telehealth services, skill development opportunities, and the flexibility to select shifts that suit their expertise and availability.We aspire to align incentives and tackle inefficiencies within the labor market, aiming to disrupt the $650 billion staffing and recruiting industry. Supported by Khosla Ventures and recognized by the Inc. 5000 for remarkable growth, WorkWhile is at the forefront of reshaping the future of work.Role OverviewWe are looking for a talented Senior Data Engineer to join our Engineering team and spearhead the development and maintenance of our robust data platform, which drives analytics, machine learning models, and operational intelligence throughout the WorkWhile staffing marketplace. In this pivotal role, you will design, build, and maintain data pipelines and infrastructure on the Google Cloud Platform, empowering our Data Science team to efficiently train and deploy models at scale, including demand forecasting, worker reliability prediction, dynamic pricing, and retention modeling.You will take ownership of our Dagster-based orchestration platform, manage our BigQuery data warehouse, and oversee the integrations connecting our PostgreSQL production database with our analytical framework. Collaborating closely with Data Science, Product Engineering, and Operations teams, you will ensure that reliable, well-structured, high-quality data is accessible when and where it’s needed. Your work will significantly enhance how WorkWhile matches workers with opportunities, anticipates marketplace dynamics, and improves overall experiences for both workers and businesses.Join us in a dynamic, collaborative environment where your contributions are crucial to our mission of making hourly work more accessible, dependable, and rewarding for everyone.

About RoxAt Rox, we are dedicated to empowering individuals to excel in their work.Our innovative platform enhances seller performance with autonomous revenue agents that handle manual tasks, allowing sales professionals to concentrate on their primary role: selling. Just as coding agents exponentially increased engineering productivity, our revenue agents elevate customer engagement.We are revolutionizing the revenue landscape by developing the world’s first revenue operating system, integrating everything from the application layer to the system of context. At Rox, we envision a future where humans become orchestrators, while our agents efficiently manage the complete customer lifecycle.Rox serves Global 2000 leaders across sectors such as banking, hardware, construction, and sovereign AI, while also supporting prominent AI innovators like Ramp and Cognition.This success is built on a steadfast belief in our mission and an unwavering commitment to achieving it.Join Our TeamOur world-class team is the cornerstone of our success, tasked with redefining business operations.Team members have:Founded and exited successful startupsHeld top positions at Google, AWS, Confluent, and New RelicAchieved gold medals in IMO and IOIPublished groundbreaking research papersWe are proud to be backed by top-tier investors, having raised $50M from Sequoia (Alfred Lin), General Catalyst (Hemant Taneja), Google Ventures, Elad Gil, and Chris Ré.Core PrinciplesTaste: Craft exceptional experiences.Every detail matters; we ensure that each interaction is optimized for the seller’s success and satisfaction. We strive for perfection and continuously seek ways to delight our users.Obsession: Commit without reservation.We are dedicated to our craft, proactively addressing customer needs and constantly improving our skills and products.Action: Deliver results.Execution is key; we prioritize thoughtful, rapid decision-making and immediate delivery, fostering a culture of trust.

Join our dynamic Engineering team as a Senior Data Engineer and play a pivotal role in shaping Tendo’s innovative data engineering solutions. You will be responsible for ingesting, transforming, and warehousing critical healthcare-related data from diverse sources.In this collaborative environment, you will work closely with our Data Scientists, Product Managers, and Machine Learning Engineers to create robust data flows and transformations that facilitate advanced analytics and the development of AI/ML models.Your expertise will be instrumental in designing and implementing tools and solutions that streamline data integration, data warehousing, and data modeling. This role enables Data Engineers and Data Scientists to experiment and train machine learning models, leading to valuable insights for Tendo’s clients.The ideal candidate will possess a robust background in software engineering, data modeling, data warehousing, ETL pipelines, and database design. A keen interest or experience in AI/ML applications, along with a forward-thinking approach to healthcare innovation, is highly desirable.

At Point, we're redefining the banking landscape by transforming the everyday debit card into a gateway for rewards and benefits. Join our innovative team as we embark on this exciting journey!We are seeking a skilled Data Engineer to lead our data initiatives at Point. In this role, you will be responsible for designing and implementing robust data pipelines and systems that will support the development of cutting-edge algorithms and models. You will have full ownership of our data infrastructure while establishing the groundwork for data engineering practices at Point. Collaborating closely with both the engineering and data science teams, you will play a pivotal role in driving product initiatives and achieving our business objectives. This position reports directly to the CTO.Discover more about Point's vibrant culture and dedicated team here.

Okta is an independent identity provider focused on building secure, trusted infrastructure for both AI and human users. The Technology, Data, and Intelligence (TDI) team supports Okta’s global workforce by providing the technology and systems employees need to succeed. Role overview The Senior Director of Data Platform and Engineering leads a global group of data and analytics engineers. This leader maximizes the value of Okta’s data assets and reports to the VP of Data and Insights. The position requires a balance of deep technical expertise and the ability to engage in strategic business discussions. As a player-coach, the Senior Director builds and mentors the team, guides key initiatives, and maintains a strong data foundation. This role is central to Okta’s AI strategy. As AI becomes more integrated into Okta’s products and operations, the quality and governance of data are increasingly important. The Senior Director ensures that clean, trusted, and well-managed data supports all AI projects, shaping the platform for Okta’s ongoing growth. What you will do Lead and develop a high-performing team: Mentor, grow, and support a diverse group of data and analytics engineers. Foster a culture of excellence, collaboration, and continuous learning. Advance AI enablement: Work closely with AI Engineering teams to ensure data infrastructure and practices are ready for AI development and deployment. Define data governance standards, build quality training datasets, and develop scalable data pipelines for AI and machine learning models. Location Bellevue, Washington; Chicago, Illinois; San Francisco, California; Washington, DC

About UsAt Imprint, we are transforming the landscape of co-branded credit cards and financial products to be more innovative, rewarding, and brand-centric. We collaborate with distinguished companies such as Crate & Barrel, Rakuten, Booking.com, H-E-B, Fetch, and Brooks Brothers to launch contemporary credit programs that enhance customer loyalty, unlock savings, and stimulate growth. Our platform seamlessly integrates advanced payment infrastructure, intelligent underwriting, and a user-friendly experience, enabling brands to offer impactful financial products without the need to operate as a bank.Co-branded cards represent over $300 billion in annual spending in the U.S., yet many are still reliant on outdated banking systems. Imprint stands out as the modern alternative: adaptable, technology-driven, and tailored for today’s consumers. Supported by renowned investors such as Kleiner Perkins, Thrive Capital, and Khosla Ventures, we are assembling a top-tier team to revolutionize payment methods and foster brand growth. If you're eager to work in a dynamic environment, tackle challenging problems, and make a significant impact, we invite you to join us.Our TeamThe Data Engineering team at Imprint is tasked with developing and expanding the data infrastructure that underpins product innovation, analytics, operations, and machine learning throughout the organization. We are responsible for creating the pipelines, platforms, and processes that enable our stakeholders to trust and leverage data effectively.We are seeking a Data Engineer to advance our modern data architecture and deliver dependable, scalable data solutions. Your contributions will directly influence decision-making and innovation across various business areas—from financial operations to real-time personalization.

Are you ready to challenge your limits and push the boundaries of your capabilities? Join us at Coinbase, where we strive to enhance economic freedom globally. As we progress in building the future of the on-chain platform and the global financial ecosystem, we need the brightest minds to help us achieve this monumental goal.We're looking for a dedicated Senior Data Protection Engineer who shares our passion for transforming the financial landscape through blockchain technology. If you are motivated to make a significant impact, thrive under pressure, and value continuous feedback to grow, you will find your place among our talented team at Coinbase.Our work culture is both demanding and rewarding, designed for those who are eager to excel alongside high-performing colleagues. Although many of our roles are remote-first, we do emphasize the importance of in-person collaboration throughout the year, with team and company-wide events held multiple times annually to strengthen our connections and alignment.As a Senior Data Protection Engineer, you will spearhead the enhancement of our data protection capabilities, ensuring that the Coinbase ecosystem is shielded from sophisticated security threats. Your primary responsibility will be executing a comprehensive strategy that harmonizes strong security measures with the agility required in a decentralized tech environment. You will adopt an automation-first approach, utilizing advanced technologies such as LLMs and agentic AI to create scalable Data Loss Prevention operations. Collaborating across various teams will be essential to mitigate risks and ensure compliance with global regulations.

Join our dynamic team at Cargomatic as a Senior Data Architect in Data Engineering, where you'll play a pivotal role in designing and constructing scalable, cloud-native data infrastructures. Your work will empower analytics, machine learning, and AI-driven applications that revolutionize the local trucking industry.In this fast-paced environment, you'll leverage your deep expertise in data architecture alongside hands-on experience with modern data platforms and LLM-enabled application development. You will be responsible for leading the design of enterprise-grade data models, architecting RAG systems, implementing agentic workflows, and integrating secure, production-ready LLM capabilities into our ecosystem. This high-impact position offers significant ownership and visibility, allowing you to shape the future of intelligent logistics technology.

At Judgment Labs, we specialize in developing cutting-edge infrastructure for Agent Behavior Monitoring (ABM). Unlike conventional observability tools that merely track exceptions and latency, our ABM technology identifies behavioral anomalies, such as instruction drifts and context retrieval losses, in large-scale production settings.Our solutions are trusted by numerous teams working on autonomous agents to gain insights into system behavior post-deployment. Rather than simply reacting to incidents, our clients analyze patterns across conversations and workflows, correlate regressions with specific interaction types, and identify critical points of reliability failure. Recently, we secured over $30 million across two funding rounds from notable investors like Lightspeed, SV Angel, and Valor Equity Partners.The Role:We are seeking a Senior Data Infrastructure Engineer to architect and enhance the real-time data pipelines essential for robust agent behavior analysis at scale. This position plays a vital role in processing hundreds of thousands of traces per second, executing LLM-based scoring and clustering in near-real-time, and ensuring low-latency query performance, which allows teams to monitor agent behavior as it unfolds. Ideal candidates will have experience designing petabyte-scale data systems, optimizing OLAP database performance, and managing the full data lifecycle from ingestion to analytics.What You'll Do:Design and automate large-scale, high-performance streaming and batch data processing systems to support Judgment's behavioral analysis products.Collaborate closely with infrastructure and backend teams to enhance scalability, data governance, and operational efficiency.Promote best practices in software engineering for data infrastructure at scale.Uphold high standards for data quality and engineering: ensuring reliability, efficiency, documentation, testability, and maintainability.Craft data models for optimal storage and access, ensuring efficient data flows to meet critical product requirements.Enhance OLAP database performance through careful schema design, partitioning strategies, storage optimization, and access pattern analysis.

At Plaid, we believe in the power of data-driven decision-making. Our data culture demands robust and scalable data systems that ensure accuracy and completeness. As a Senior Software Engineer focusing on Data Infrastructure, you will play a pivotal role in empowering teams across engineering, product, and business sectors to swiftly and securely extract valuable data insights. Your work will directly enhance our ability to serve customers effectively. You will be responsible for building and optimizing our data and machine learning infrastructure, allowing Plaid engineers to innovate and iterate on products built on consumer-permissioned financial data. Our Data Infrastructure engineers are experts in Data Warehousing, Data Lakehouse architecture, Spark, Workflow Orchestration, and Streaming technologies. You will enhance our existing data pipelines for performance and cost efficiency while creating intuitive abstractions that simplify the development process for other engineers at Plaid.

About Us:At Notion, we empower individuals and teams to create stunning productivity tools tailored to their unique workflows. In a digital landscape crowded with applications and tabs, our platform unifies documents, notes, projects, calendars, and emails—all enhanced with AI capabilities to streamline tasks and provide quick insights. Trusted by millions, including industry leaders like Toyota, Figma, and OpenAI, Notion is chosen for its versatility, helping users optimize their time and resources.We believe in the value of in-person collaboration, making it a cornerstone of our company culture. To foster teamwork, all employees are required to work from our San Francisco office on Mondays, Tuesdays, and Thursdays—our designated Anchor Days. Certain roles may necessitate additional in-office presence.About The Role:As our company expands at an impressive pace, we are on the lookout for skilled Data Engineers to join our finance team. In this role, you will be instrumental in constructing robust datasets and data pipelines that underpin our financial reporting. Your work will bridge our product, financial, and operational systems, creating reliable processes that drive our growth. If you have a passion for data analytics, complex modeling, and innovative problem-solving, we would love to have you on board.What You'll Achieve:Develop essential datasets that act as the definitive sources for Notion’s financial reporting, integrating information from financial systems, business databases, and our product.Collaborate closely with our Finance, Monetization Engineering, Business Intelligence, and Data Science teams to address critical financial analysis and reporting requirements.Design, construct, and oversee data pipelines that not only meet current demands but are also scalable as our data needs grow.Facilitate widespread access to high-quality financial data across the Finance, Staff, and go-to-market teams.Skills You'll Need to Bring:A minimum of 4 years of experience as a data engineer, focusing on building core datasets and supporting various business sectors, particularly in product and high-volume data environments. A strong enthusiasm for analytics use cases, data modeling, and tackling intricate data challenges is essential.Experience in creating integrations and reporting datasets for payment, financial, and business systems such as Stripe, NetSuite, Adaptive, Anaplan, Salesforce, or equivalent.A proactive mindset, with the ability to gather and synthesize high-impact needs from business partners, designing and implementing suitable technical solutions.

At Databricks, we are driven by a passion for empowering data teams to tackle the world’s most challenging problems — from transforming transportation to accelerating medical innovations. We achieve this by creating and maintaining the leading data and AI infrastructure platform, enabling our clients to leverage profound data insights for business enhancement. Founded by engineers with a customer-first mentality, we eagerly embrace every opportunity to tackle complex technical challenges, ranging from the design of next-generation UI/UX for data interactions to scaling our services across millions of virtual machines. Our journey has just begun.As a member of the Runtime team at Databricks, you will be instrumental in developing the next generation of distributed data storage and processing systems. These systems will surpass specialized SQL query engines in relational query performance while offering the programming abstractions necessary to support a variety of workloads, from ETL to data science.Example projects include:Apache Spark™: Contribute to the de facto open-source standard framework for big data.Data Plane Storage: Develop reliable and high-performance services and client libraries for managing vast amounts of data within cloud storage backends like AWS S3 and Azure Blob Store.Delta Lake: Design a storage management system that merges the scalability and cost-effectiveness of data lakes with the performance and reliability of data warehouses, providing features like ACID transactions and time travel.Delta Pipelines: Simplify the orchestration and operation of numerous data pipelines, enabling clients to deploy, test, and upgrade pipelines effortlessly.Performance Engineering: Create the next-generation query optimizer and execution engine that is fast, scalable, and robust.