Databricks Data Engineer

We are looking for a certified Databricks Data Engineer who will turn data into information, information into insight, and insight into business decisions. This is a unique opportunity to be one of the key drivers of change in our expanding company.

Work at Exadel - Who We Are 

Since 1998, Exadel has been engineering its products and custom software for clients of all sizes. Headquartered in Walnut Creek, California, Exadel has 2,800+ employees in development centers across America, Europe, and Asia. Our people drive Exadel’s success and are at the core of our values.

About the Customer

The company is headquartered in New York City, has 60+ offices in the Americas, Europe, Asia Pacific, and the Middle East, and employs 4,500+ professionals. This international management consulting company focuses on retail services.

Requirements

• 4+ years of background in Data Engineering with hands-on work using Databricks
• Strong expertise in ETL processes and data pipeline development
• Proficiency with Spark and Python for large-scale data processing in Databricks
• Experience with data extraction from web-based sources (APIs, web scraping, or similar approaches)
• Familiarity with handling structured and unstructured data and ensuring efficient data storage and access
• Competency in Azure or other cloud platforms
• Knowledge of SQL and database management for data validation and storage
• Problem-solving skills with the ability to optimize processes for periodic updates
• Communication skills and the ability to work in a collaborative environment

Nice to have

• Experience with Azure Data Factory or similar tools for orchestrating and automating data pipelines
• Knowledge of data management in AWS or GCP environments
• Familiarity with API integration for data extraction

English level

Upper-Intermediate

Responsibilities

• Design, develop, and implement data ingestion and transformation pipelines using Databricks and Azure Databricks
• Manage and orchestrate data extraction from a public information website, handling bulk data downloads efficiently
• Develop solutions for periodic data updates (monthly), optimizing data refresh cycles to ensure accuracy and efficiency
• Clean, aggregate, and transform the data for analysis, ensuring the quality and completeness of data
• Collaborate with stakeholders to understand data requirements and propose solutions for efficient data management
• Implement best practices for ETL processes in a Databricks environment, ensuring scalability and performance
• Monitor and troubleshoot data pipelines, ensuring smooth operation and timely updates.
• Document the data engineering processes, workflows, and architecture

Advantages of Working with Exadel

Transparent Career Roadmap

• The HR manager accompanies each employee on their first steps in the company and assists them in achieving future professional goals
• Exadel is a global company with 30+ offices in 15+ countries. Employees have the chance to discover new places to live and work and can transfer to any location that has an Exadel production office. The HR manager and the Global Mobility specialist accompany the employee and assist throughout the relocation process
• Your career growth is in your hands, so each Exadeller decides when to level up and initiate performance assessments. The management team supports career planning and the discovery of new tech stacks and languages

Learning & Development Opportunities

• We believe in nurturing talent and fostering growth. That's why we budget every year for your continuous learning journey. From professional development courses to books and certification, we're invested in your success.
• Employees participate actively and benefit from the knowledge-sharing environment across seven practice units: Digital Marketing Technologies, Content Marketing, AI, QA, QAA, Agile, and DevOps
• Employees can find recordings from meetups and trainings on Exadel’s Internal Learning System

Work & Life Balance

• Each Exadeller is in charge of managing their workflow independently without being micromanaged at every step; it is, however, important that an employee always attends meetings, logs their time, and meets deadlines
• Operating from the office or remotely depends on which mode the employee prefers. Hybrid or remote mode staff will receive the necessary equipment before their first working day. For those who prefer to work in the office, the office network  offers up-to-date equipment, good facilities, and a warm, collaborative environment
• The team meets at two annual team-building events across locations; in addition, there are team events such as Pizza Days, Programmer’s Day, informal meetups, board game parties, and sports initiatives

Employee Perks & Bonuses

• The Exadel referral program covers all regions and involves recognition for hired recommended professionals. Depending on the location and the position, an employee receives a monetary bonus for recommending a candidate to Exadel who ends up being hired
• Employees collect coins for numerous project activities, exchanging them for products or services of their choice available at the Exadel store
• Volunteers for mentoring, interviewing, performing assessments, and participating in community meet-ups get additional monetary bonuses
• Location-dependent, Exadellers can access benefit platforms with vouchers for online stores, medical insurance reimbursement, sports membership fees, vacations, and childbirth bonuses