About the job
Role: Senior Cloudera Developer (Data Engineer)
Experience: 6 to 10 Years
Location: Pune
Job Description:
- Expertise in Spark programming and architecture, with a solid understanding of fault tolerance mechanisms.
- Proficient in utilizing Spark DataFrames and Spark SQL for querying structured datasets.
- Experience in optimizing Spark execution plans is advantageous.
- Skilled in performing Extract, Transform, Load (ETL) processes using Spark.
- Experience integrating Spark Streaming with technologies like Kafka is a plus.
- Familiarity with the Hadoop ecosystem, including HDFS, Hive, and Cloudera stack, is beneficial.
- Experience in deploying and managing Spark applications on Hadoop clusters or GCP Dataproc.
- Strong proficiency in Python, with experience in Java being a bonus.
- Familiar with DevOps tools and practices, including CI/CD and Docker.
- Hands-on experience with GCP services such as Dataproc, Cloud Functions, Cloud Run, Pub/Sub, and BigQuery.
Responsibilities:
- Design and implement data solutions leveraging Cloudera technologies like Hadoop, Spark, and Hive.
- Collaborate with data engineering teams to enhance data pipelines and processing workflows.
- Work closely with data analysts and scientists to ensure data quality and integrity.
- Diagnose and resolve issues related to data processing and storage systems.
- Stay abreast of the latest developments and best practices in Cloudera development.
- Participate in code reviews and contribute constructive feedback to peers.

