Qualifications
Required qualificationsA minimum of 4 years of hands-on experience in data engineering. Extensive experience with Apache Airflow, AWS Glue, PySpark, and Python-based data pipelines. Strong command of SQL and experience with PostgreSQL in live operational environments. Solid understanding of cloud-native data workflows, preferably on AWS, as well as pipeline observability including metrics, logging, tracing, and alerting. Demonstrated experience managing pipelines end-to-end: from design and implementation to testing, deployment, monitoring, and iteration. Preferred qualificationsExperience optimizing Snowflake performance (e.g., warehouses, partitions, clustering, query profiling) and cost efficiency. Experience with real-time or near-real-time processing techniques (e.g., streaming ingestion, incremental models, CDC). Familiarity with backend TypeScript frameworks (e.g., NestJS) is a significant advantage. Experience with data quality frameworks, contract testing, or schema management (e.g., Great Expectations, dbt tests, OpenAPI/Protobuf/Avro). A background in developing internal developer platforms or data platform components (such as connectors, SDKs, CI/CD processes for data).
About the job
We are on the lookout for an exceptional Data Engineer, a technical leader who thrives on challenges and excels in coding. If you are the person who:
Acts as the definitive technical authority within your team
Solves complex technical problems with ease
Delivers intricate features at a remarkable speed
Writes code that exemplifies best practices and clarity
Is dedicated to enhancing the overall quality of the codebase
Then we want to hear from you!
We are not looking for just anyone; we want developers who are confident in their skills and have proven their excellence.
What you will be responsible for:
Designing, optimizing, and expanding data pipelines and infrastructure leveraging Python, TypeScript, Apache Airflow, PySpark, AWS Glue, and Snowflake.
Creating, operationalizing, and monitoring data ingestion and transformation workflows including DAGs, alerting mechanisms, retries, SLAs, lineage, and cost management.
Partnering with platform and AI/ML teams to streamline ingestion, validation, and real-time compute workflows; contributing towards the development of a feature store.
Incorporating pipeline health metrics into engineering dashboards to ensure complete visibility and observability.
Modeling data and executing efficient, scalable transformations within Snowflake and PostgreSQL.
Establishing reusable frameworks and connectors to standardize internal data publishing and consumption.
About pridelogic
pridelogic is a pioneering technology firm that specializes in delivering innovative data solutions. Our team is driven by a commitment to excellence and a passion for crafting top-tier software that empowers organizations to harness the full potential of their data. We foster a collaborative and inclusive environment where every team member's skills are valued.