Qualifications
RequirementsKey ResponsibilitiesLead the evolution of our Data Platform by enhancing Arkham’s Data Platform in alignment with Lakehouse architecture principles while ensuring robust data governance. Design and build Data Ingestion Pipelines that extract data from structured, semi-structured, and unstructured sources. Create and optimize Data Pipeline Orchestration for monitoring and managing multiple extraction and transformation pipelines. Integrate Data Catalogs to ensure seamless interoperability among various query engines. Oversee Cluster Management & Observability to maintain optimal performance of data pipelines. Ensure End-to-End Data Lifecycle Management by maintaining high data quality and usability throughout integration, transformation, and activation stages. QualificationsA minimum of 5 years of experience in data engineering, data architecture, or a related field. Demonstrated technical expertise in Apache Spark, Delta Lake, and Trino. Strong programming skills in Python for scripting and automation tasks. Hands-on experience with AWS services, including Glue, S3, and EMR. Solid understanding of distributed data systems and query engines. Exhibit exceptional analytical and debugging skills for effective problem-solving.
About the job
About Arkham Technologies
Arkham Technologies is a pioneering Data & AI Platform that offers a comprehensive suite of robust tools aimed at unifying your data and applying advanced Machine Learning and Generative AI models to tackle your most intricate operational challenges. Our platform is trusted by industry leaders, including Circle K, Mexico Infrastructure Partners, and Televisa Editorial, to streamline data access, automate complex workflows, and enhance operational efficiency. By leveraging our platform and implementation services, clients benefit from significant time savings, cost reductions, and a solid foundation for sustainable Data and AI transformation.
About the Role
We are seeking a highly skilled Senior Data Engineer to take ownership of our high-performance Data Platform, built on the innovative Lakehouse architecture. In this pivotal role, you will engage with cutting-edge technologies such as Apache Spark, Trino, and Delta Lake, ensuring data governance and interoperability across various platforms. You will significantly influence our data infrastructure, managing the entire data lifecycle from ingestion through transformation to activation.
About Arkham Technologies
Arkham Technologies is at the forefront of the data revolution, providing advanced solutions that empower organizations to harness their data effectively. Our innovative platform supports a wide array of clients in optimizing their operations and making informed decisions through data-driven insights.