Kafka Data Engineer

Job Title: Kafka Data Engineer
Location: Bangalore/Chennai India
Employment Type: Full-Time

Role Overview:
We are seeking a skilled Kafka Data Engineer to design, build, and optimize real-time data streaming solutions using Apache Kafka and Google Cloud Platform (GCP). The ideal candidate will have hands-on experience in Kafka topic design, data pipeline development, and cloud-native data engineering practices.

Key Responsibilities:
- Design and implement scalable and secure Kafka-based data streaming solutions.
- Develop and maintain real-time and batch data pipelines using GCP services such as Dataflow,Pub/Sub, and BigQuery.
- Collaborate with architects, data engineers, and DevOps teams to ensure seamless integration of Kafka with cloud data platforms.
- Optimize Kafka topic design, partitioning strategies, and message schemas for performance and reliability.
- Monitor and troubleshoot Kafka clusters and streaming jobs to ensure high availability and low latency.
- Implement best practices for data quality, governance, and security in streaming environments.
- Participate in code reviews, design discussions, and continuous improvement initiatives.

Required Qualifications:
- 5+ years of experience in data engineering with a focus on real-time data processing.
- Strong hands-on experience with Apache Kafka, including topic design, schema registry, and Kafka Streams.
- Proficiency in GCP services such as Pub/Sub, Dataflow, BigQuery, and Cloud Storage.
- Solid understanding of ETL/ELT processes, data modeling, and distributed systems.
- Experience with CI/CD pipelines and infrastructure-as-code tools like Terraform.
- Strong problem-solving skills and the ability to work in a collaborative, agile environment.

Preferred Qualifications:
- GCP certifications (e.g., Professional Data Engineer).
- Experience with DevOps practices and containerization (Docker, Kubernetes).
- Familiarity with multi-cloud environments and hybrid data architectures.