companypridelogic logo

Senior Data Engineer (Python/PySpark/AWS Glue/Amazon Athena/SQL/Apache Airflow)

pridelogicRemote job
Remote Full-time

Clicking Apply Now takes you to AutoApply where you can tailor your resume and apply.


Unlock Your Potential

Generate Job-Optimized Resume

One Click And Our AI Optimizes Your Resume to Match The Job Description.

Is Your Resume Optimized For This Role?

Find Out If You're Highlighting The Right Skills And Fix What's Missing

Experience Level

Mid to Senior

Qualifications

Required qualificationsA minimum of 4 years of hands-on experience in data engineering. Extensive experience with Apache Airflow, AWS Glue, PySpark, and Python-based data pipelines. Strong command of SQL and experience with PostgreSQL in live operational environments. Solid understanding of cloud-native data workflows, preferably on AWS, as well as pipeline observability including metrics, logging, tracing, and alerting. Demonstrated experience managing pipelines end-to-end: from design and implementation to testing, deployment, monitoring, and iteration. Preferred qualificationsExperience optimizing Snowflake performance (e.g., warehouses, partitions, clustering, query profiling) and cost efficiency. Experience with real-time or near-real-time processing techniques (e.g., streaming ingestion, incremental models, CDC). Familiarity with backend TypeScript frameworks (e.g., NestJS) is a significant advantage. Experience with data quality frameworks, contract testing, or schema management (e.g., Great Expectations, dbt tests, OpenAPI/Protobuf/Avro). A background in developing internal developer platforms or data platform components (such as connectors, SDKs, CI/CD processes for data).

About the job

We are on the lookout for an exceptional Data Engineer, a technical leader who thrives on challenges and excels in coding. If you are the person who:

  • Acts as the definitive technical authority within your team

  • Solves complex technical problems with ease

  • Delivers intricate features at a remarkable speed

  • Writes code that exemplifies best practices and clarity

  • Is dedicated to enhancing the overall quality of the codebase

Then we want to hear from you!
We are not looking for just anyone; we want developers who are confident in their skills and have proven their excellence.

What you will be responsible for:

  • Designing, optimizing, and expanding data pipelines and infrastructure leveraging Python, TypeScript, Apache Airflow, PySpark, AWS Glue, and Snowflake.

  • Creating, operationalizing, and monitoring data ingestion and transformation workflows including DAGs, alerting mechanisms, retries, SLAs, lineage, and cost management.

  • Partnering with platform and AI/ML teams to streamline ingestion, validation, and real-time compute workflows; contributing towards the development of a feature store.

  • Incorporating pipeline health metrics into engineering dashboards to ensure complete visibility and observability.

  • Modeling data and executing efficient, scalable transformations within Snowflake and PostgreSQL.

  • Establishing reusable frameworks and connectors to standardize internal data publishing and consumption.

About pridelogic

pridelogic is a pioneering technology firm that specializes in delivering innovative data solutions. Our team is driven by a commitment to excellence and a passion for crafting top-tier software that empowers organizations to harness the full potential of their data. We foster a collaborative and inclusive environment where every team member's skills are valued.

Similar jobs

Tailoring 0 resumes

We'll move completed jobs to Ready to Apply automatically.