Qualifications
Key Responsibilities:Technical Leadership & Architecture: Lead the implementation of new Python-based ETL pipelines for data movement from Luna’s microservice DynamoDB tables to the RDS transactional database (Postgres OLTP) and subsequently to the RDS data warehouse (Postgres OLAP). Platform Optimization: Utilize your previous experiences to enhance the performance of the new Postgres OLTP for transactional efficiency and the OLAP environment for analytical effectiveness, all while managing costs effectively. Standard Setting: Establish and uphold DataOps standards, including version control (Git), automated CI/CD deployment, and schema migration management using tools such as Liquibase or Drizzle ORM. Team Enablement & Mentorship:Hands-on Coaching: Provide active mentorship to team members, enhancing their expertise in Python, Git, and engineering workflows through code reviews and workshops. Code Quality: Conduct thorough code reviews, offering detailed, educational feedback that elucidates best practices and necessary changes to elevate team standards. AI Engineering Patterns: Assist in defining and implementing AI tools and practices to augment the Data Engineering team. Strategic Agile Delivery:Initiative Planning: Decompose larger reporting initiatives into manageable epics and stories within our Agile framework.
About the job
About Us:
At AND Digital, we are a pioneering tech company committed to accelerating digital delivery and bridging the digital skills divide. Since our inception in 2014, we have empowered organizations to create superior digital products and build robust digital teams.
Our philosophy is rooted in leaving a lasting legacy for our clients by fostering close-knit relationships through our regional 'Clubs', ensuring our partners are prioritized by dedicated teams in their vicinity.
This distinctive approach has fueled remarkable organic growth for both our clients and ourselves, and we invite you to join us in our mission to close the global digital skills gap.
Position Overview:
We are in search of a Lead Data Engineer to serve as the technical practice lead for our Luna data team. This pivotal role involves spearheading a high-priority migration from our existing Redshift environment to a new AWS RDS (Postgres) architecture.
In addition to overseeing technical architecture, you will mentor and develop the skills of our Data Analysts, guiding their transition from SQL-centric workflows to production-ready Python and DataOps methodologies, while collaborating with stakeholders to convert business requirements into scalable technical roadmaps.