Back to jobs
Lead Data Engineer (DBT, Databricks, Azure)
Latin America
Job Title: Lead Data Engineer (DBT, Databricks, Azure)
Key Skills: Python, REST APIs, Azure Databricks, DBT, SQL, Terraform, Data Engineering, Data Modeling, CI/CD, Team Leadership
Experience: 8+ years of experience.
Location: Open to candidates across LATAM.
At Coforge, we are looking for a Lead Data Engineer (#22145) with the following profile.
Responsibilities
• Lead the RTS delivery squad and own the end-to-end delivery of strategic data products within a modern cloud data platform.
• Design and oversee the architecture of low-latency, API-driven data solutions, from backend data models through serving layers and API consumption.
• Build and maintain scalable backend services and REST APIs using Python and object-oriented programming principles.
• Design, develop, and optimize data pipelines using Azure Databricks, DBT, SQL, and cloud-native technologies.
• Own the architecture and implementation of curated data layers, ensuring high availability, scalability, and performance.
• Evaluate, select, and implement technologies for low-latency data serving and API consumption.
• Lead infrastructure deployment and management using Terraform and Infrastructure as Code (IaC) practices.
• Mentor and develop a team of data and machine learning engineers, providing technical guidance, coaching, and career development support.
• Conduct code reviews and establish engineering standards, development best practices, and architectural guidelines.
• Collaborate with Product, Engineering, Business, and Project stakeholders to define technical solutions and delivery roadmaps.
• Ensure data quality, observability, monitoring, alerting, and SLA compliance across all data products and services.
• Drive CI/CD adoption and oversee release management processes for pipelines, services, and platform components.
• Act as the primary technical escalation point for troubleshooting complex production issues and ensuring long-term platform stability.
• Maintain comprehensive technical documentation, architectural decisions, and implementation standards.
• Design and oversee the architecture of low-latency, API-driven data solutions, from backend data models through serving layers and API consumption.
• Build and maintain scalable backend services and REST APIs using Python and object-oriented programming principles.
• Design, develop, and optimize data pipelines using Azure Databricks, DBT, SQL, and cloud-native technologies.
• Own the architecture and implementation of curated data layers, ensuring high availability, scalability, and performance.
• Evaluate, select, and implement technologies for low-latency data serving and API consumption.
• Lead infrastructure deployment and management using Terraform and Infrastructure as Code (IaC) practices.
• Mentor and develop a team of data and machine learning engineers, providing technical guidance, coaching, and career development support.
• Conduct code reviews and establish engineering standards, development best practices, and architectural guidelines.
• Collaborate with Product, Engineering, Business, and Project stakeholders to define technical solutions and delivery roadmaps.
• Ensure data quality, observability, monitoring, alerting, and SLA compliance across all data products and services.
• Drive CI/CD adoption and oversee release management processes for pipelines, services, and platform components.
• Act as the primary technical escalation point for troubleshooting complex production issues and ensuring long-term platform stability.
• Maintain comprehensive technical documentation, architectural decisions, and implementation standards.
Mandatory Requirements
• 8+ years of experience in Data Engineering, Software Engineering, or related technical roles.
• At least 3 years of experience leading engineering teams, including senior, mid-level, and junior professionals.
• Strong backend development experience with proven delivery of production-grade REST APIs and backend services.
• Expert-level programming skills in Python, including strong object-oriented programming (OOP) principles.
• Extensive experience with PyTest, unit testing, automated testing frameworks, and software quality practices.
• Strong hands-on experience with Azure Databricks and DBT.
• Advanced SQL skills and experience building scalable data solutions.
• Deep understanding of data modeling, dimensional modeling, and consumption-oriented data architectures.
• Extensive experience with Terraform and Infrastructure as Code (IaC) deployments.
• Experience designing and delivering low-latency data products, serving layers, or API-driven data solutions.
• Strong knowledge of CI/CD pipelines, Git-based version control, and modern software engineering practices.
• Proven ability to lead technical delivery while remaining hands-on with architecture, development, and code reviews.
• Excellent communication and stakeholder management skills, with the ability to work effectively across technical and business teams.
• At least 3 years of experience leading engineering teams, including senior, mid-level, and junior professionals.
• Strong backend development experience with proven delivery of production-grade REST APIs and backend services.
• Expert-level programming skills in Python, including strong object-oriented programming (OOP) principles.
• Extensive experience with PyTest, unit testing, automated testing frameworks, and software quality practices.
• Strong hands-on experience with Azure Databricks and DBT.
• Advanced SQL skills and experience building scalable data solutions.
• Deep understanding of data modeling, dimensional modeling, and consumption-oriented data architectures.
• Extensive experience with Terraform and Infrastructure as Code (IaC) deployments.
• Experience designing and delivering low-latency data products, serving layers, or API-driven data solutions.
• Strong knowledge of CI/CD pipelines, Git-based version control, and modern software engineering practices.
• Proven ability to lead technical delivery while remaining hands-on with architecture, development, and code reviews.
• Excellent communication and stakeholder management skills, with the ability to work effectively across technical and business teams.
Preferred Requirements
• Experience within insurance, financial services, or other highly regulated industries.
• Knowledge of insurance concepts such as policy administration, claims, underwriting, and customer data.
• Experience with Snowflake and Snowflake-to-Databricks integration or migration patterns.
• Hands-on experience with Azure AI Search, Lakebase, feature stores, or similar low-latency serving technologies.
• Experience with streaming and real-time data technologies such as Kafka, Event Hubs, or Delta Live Tables.
• Experience implementing data governance, security, and compliance frameworks.
• Experience managing large-scale DBT environments and terabyte-scale data pipelines.
• Experience leading cloud migration initiatives, particularly AWS-to-Azure migrations.
• Familiarity with machine learning platforms and collaboration with ML engineering teams.
• Knowledge of insurance concepts such as policy administration, claims, underwriting, and customer data.
• Experience with Snowflake and Snowflake-to-Databricks integration or migration patterns.
• Hands-on experience with Azure AI Search, Lakebase, feature stores, or similar low-latency serving technologies.
• Experience with streaming and real-time data technologies such as Kafka, Event Hubs, or Delta Live Tables.
• Experience implementing data governance, security, and compliance frameworks.
• Experience managing large-scale DBT environments and terabyte-scale data pipelines.
• Experience leading cloud migration initiatives, particularly AWS-to-Azure migrations.
• Familiarity with machine learning platforms and collaboration with ML engineering teams.
Published on: 25-08-2026
At Coforge, we hire professionals solely based on their skills and qualifications and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.
Apply for this job
*
indicates a required field