Lead Data Engineer

Experience: 7+ Years

Location: Chennai / Bangalore / Coimbatore/ Pune (Hybrid)

Job Summary:

We are looking for a highly skilled Senior Data Engineer with strong expertise in Google Cloud Platform (GCP), Apache Beam/Dataflow, Java, and BigQuery. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines and cloud-based data processing solutions for enterprise applications. This role requires hands-on experience in building high-performance batch and real-time data processing systems while ensuring data quality, scalability, and reliability.

Key Responsibilities:

1. Design, develop, and maintain scalable ETL/ELT data pipelines using GCP services.
2. Build and optimize batch and real-time data processing pipelines using Apache Beam and Google Cloud Dataflow.
3. Develop robust backend applications and data processing solutions using Java.
4. Design and optimize BigQuery data warehouses, queries, and analytical workloads.
5. Integrate data from multiple sources including APIs, relational databases, cloud storage, and streaming platforms.
6. Optimize pipeline performance, scalability, reliability, and cloud infrastructure costs.
7. Collaborate with Architects, Data Analysts, DevOps, and Business teams to deliver end-to-end data solutions.
8. Implement data quality checks, monitoring, governance, and security best practices.
9. Troubleshoot production issues and provide long-term scalable solutions.
10. Participate in code reviews, testing, CI/CD implementation, and Agile development processes.

Required Skills:

1. 7+ years of experience in Data Engineering.
2. Strong hands-on experience with Google Cloud Platform (GCP).
3. Expertise in Google Cloud Dataflow and Apache Beam.
4. Strong programming experience in Java.
5. Hands-on experience with BigQuery for data warehousing and analytics.
6. Experience building both batch and streaming data pipelines.
7. Strong SQL skills and understanding of data modeling concepts.
8. Experience with Git, CI/CD, and Agile development methodologies.
9. Excellent analytical, debugging, and problem-solving skills.

Preferred Skills:

1. Experience with Pub/Sub, Cloud Composer, Cloud Storage, Dataproc, or Kafka.
2. Exposure to Python or Apache Spark.
3. Experience working on large-scale enterprise data platforms.
4. Knowledge of cloud performance optimization and cost management.

Educational Qualification:

1. Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
2. GCP Certification is an added advantage.

Why Join Us?

1. Work on large-scale cloud-native data engineering projects.
2. Opportunity to build modern data platforms using the latest GCP technologies.
3. Collaborative and innovation-driven work environment.
4. Hybrid work model with opportunities for professional growth.