DataBricks Developer (Digital Engineering)

Company Background

Our client is a leading life insurance company and part of a global financial services group. Founded in 1896 and headquartered in North America, the company offers a wide range of products, including annuities and life insurance, through a nationwide network of agents.

Project Description

The project involves a production system for managing clients and the life insurance programs in which they are enrolled. The specialist will be responsible for supporting the existing system and developing new features.

Technologies

• Databricks, Unity Catalog, Delta Lake, Databricks Genie
• Advanced SQL
• Python
• Microsoft Azure

What You'll Do

• Collaborate with a team of three engineers to develop and support data solutions in Databricks;
• Develop and maintain data processing workflows using Databricks, SQL, Python, and Delta Lake;
• Work with Unity Catalog to manage data access permissions, data lineage, and governance;
• Support the existing production system and participate in the implementation of new features and enhancements;
• Investigate and troubleshoot issues in Databricks data workflows and production processes;
• Collaborate with team members and stakeholders to deliver reliable data engineering solutions;

Job Requirements

• 3+ years of professional experience in data engineering;
• Strong communication and collaboration skills, with the ability to work effectively with technical and non-technical stakeholders;
• Hands-on experience designing and developing data solutions in Databricks, including Databricks architecture, Unity Catalog, Delta tables, data lineage, access permissions, Databricks Genie, and notebooks;
• Advanced SQL skills, including writing complex queries and performing data transformations;
• Proficiency in Python for data processing, automation, and development of data engineering solutions;
• Professional experience working with Microsoft Azure cloud services and data platforms;

Nice To Have

• Experience using Databricks AI capabilities, including Agent Bricks, and implementing Generative AI and RAG solutions in Databricks;
• Experience working with Azure data services, including Azure Synapse, Synapse pipelines, Azure Data Factory, and Azure SQL pools;
• Experience setting up CI/CD processes for Databricks solutions using Azure DevOps, Databricks Asset Bundles, and Infrastructure as Code practices;
• Experience with performance tuning for Databricks workloads, SQL queries, and data processing pipelines;
• Databricks Certified Data Engineer Professional certification;
• Databricks Certified Generative AI Engineer Associate certification;

What Do We Offer

The global benefits package includes:

• Technical and non-technical training for professional and personal growth;
• Internal conferences and meetups to learn from industry experts;
• Support and mentorship from an experienced employee to help you professional grow and development;
• Health insurance;
• Sports activities to promote a healthy lifestyle;
• Flexible work options, including remote and hybrid opportunities;
• Referral program for bringing in new talent;
• Work anniversary program and additional vacation days.