Data Engineer – Databricks

Build the Future with AspenView Technology Partners

At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help North American clients innovate faster and more efficiently.

As we continue to grow, we’re looking for exceptional people to join our team and help drive impactful change across industries.

Role Overview
We are seeking a skilled Data Engineer – Databricks to design, build, and optimize high-
throughput, production-grade data pipelines on the Databricks Lakehouse platform.
In this role, you will implement Medallion Architecture patterns (Bronze, Silver, Gold layers) to
process structured and unstructured healthcare data—including Electronic Health Records
(EHR/Epic), FHIR streams, claims data, and clinical research datasets. You will work closely
with data architects, clinical analysts, and cloud engineers to build secure, HIPAA-compliant
ETL/ELT pipelines using PySpark, Delta Lake, and Databricks Workflows.

Key Responsibilities
● Lakehouse Pipeline Engineering: Design, develop, and maintain automated ETL/ELT
data pipelines on Databricks utilizing PySpark, Delta Lake, and Databricks
Workflows or Delta Live Tables (DLT).
● Data Architecture Implementation: Implement Medallion Architecture principles to
clean, transform, and aggregate raw clinical/operational datasets into analytics-ready
models for BI reporting and predictive analytics.
● Healthcare Data Integration: Ingest and normalize healthcare data sources, including
Epic EHR databases (Clarity/Caboodle), FHIR/HL7 data feeds, and third-party claims
systems.
● Data Governance & Security: Enforce row/column-level access control and data
lineage using Databricks Unity Catalog, ensuring strict compliance with HIPAA,
HITRUST, and internal data governance protocols for Protected Health Information
(PHI).
● Performance Tuning & Optimization: Optimize compute cluster utilization, SQL query
performance, Delta table indexing (Z-Ordering/Partitioning), and memory management
to control cloud infrastructure costs and lower pipeline latency.
● Cross-Functional Collaboration: Partner with US-based data scientists, business
intelligence developers, and clinical stakeholders to understand data requirements and
deliver reliable data products.

Required Technical Skills
● Data Engineering Experience: 4+ years of production experience in data engineering,
with at least 2+ years of hands-on expertise building scalable solutions on Databricks.
● Databricks & Lakehouse Ecosystem: Strong mastery of PySpark, Delta Lake,
Databricks SQL, and Unity Catalog.
● Programming & Querying: Advanced proficiency in Python and SQL for complex data
manipulation and performance optimization.
● Cloud Infrastructure: Hands-on experience with managed cloud data storage and
computing services (AWS S3 / EMR or Azure ADLS Gen2 / Data Factory).
● Healthcare Interoperability (Preferred): Familiarity with data standards (FHIR, HL7,
OMOP, ICD-10/CPT codes) or EHR data structures (Epic/Cerner).
● DevOps & Quality: Experience with Git, CI/CD deployment pipelines, and data
validation frameworks (e.g., Great Expectations).
● Language Proficiency: Advanced/Fluent English (C1/C2) for daily active collaboration
with Boston-based technology and analytics teams.

Soft Skills & Competencies
● Analytical Rigor: Uncompromising attention to data quality, consistency, and structural
integrity.
● Mission-Driven Alignment: Genuine passion for applying technology and data
engineering to improve healthcare outcomes and health equity.
● Autonomous Problem Solver: Proactive approach to diagnosing pipeline failures,
identifying performance bottlenecks, and recommending architecture improvements.
● Clear Technical Communication: Ability to articulate data architecture choices and
pipeline designs clearly to technical peers and non-technical healthcare stakeholders.

Visa Sponsorship

AspenView does not sponsor employment visas for this role. Applicants must be currently authorized to work in the United States on a permanent basis without the need for visa sponsorship now or in the future.

Equal Opportunity Employer

AspenView is proud to be an equal opportunity employer. We believe in creating an environment where all employees feel welcome, valued, and empowered to succeed. We celebrate diversity and strive to build a culture of inclusion where all individuals, regardless of their race, color, gender, gender identity or expression, sexual orientation, disability, age, or any other characteristic, can thrive. We encourage applicants from all walks of life to join our team and make a lasting impact.