Data Engineer

We’re a cognitive science company on a mission to optimize the measurement of brain health to advance the development of new medicines and to enable earlier clinical insights in healthcare.

That’s why we’re seeking a Data Engineer responsible for building and maintaining data pipelines in Azure Databricks to enable Cogstate’s data strategy. The candidate will have a central critical role for establishing and maintaining the Azure Databricks infrastructure, provide best practices and recommendations for using these tools, and will working closely with data scientists, data managers, clinical scientists, and other business stakeholders as a point of contact for the data platform infrastructure.

Core Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Data Factory, and integrate data from various sources and systems into a coherent data ecosystem.
• Utilize Azure Databricks for complex data transformation and processing tasks, ensuring high performance and alignment with business requirements.
• Manage Azure Functions for efficient data ingestion and processing, optimizing for performance and cost.
• Implement and maintain data storage solutions in Azure Data Lake Storage Gen2 (ADLS2), applying best practices for data lifecycle management, security, and access control.
• Collaborate with cross-functional teams to define data requirements, deliver comprehensive data solutions, and support data-driven decision-making processes.
• Employ GitHub for source control management, adhering to best practices for code reviews, branching, and versioning.
• Facilitate CI/CD pipelines using GitHub Actions to automate testing, integration, and deployment processes, enhancing productivity and ensuring high-quality deployments.
• Monitor, troubleshoot, and optimize data systems and pipelines for performance, reliability, and scalability.
• Stay abreast of industry trends and advancements in cloud and data engineering technologies, continuously seeking opportunities for innovation and improvement.
• Maintain Data Governance infrastructure to ensure appropriate data access rights and restrictions for internal and external users.

Qualifications

• BS/BA in Computer Science, Data Science, or a related field or relevant experience
• 5+ years’ experience as a data engineer.
• Must have 2+ years experience in implementing data engineering solutions with Apache Spark and Databricks.
• Must have deep expertise in PySpark for data process and to write data pipelines and data processing layers. Experience with other technologies such as Python, Scala, Hadoop, Hive, and Spark are nice to have.
• Direct experience with relational databases like SQL Server. Excellent SQL experience for writing complex SQL transformations.
• Strong knowledge on Databricks configuration, troubleshooting and performance tuning.
• Experience with development tools for CI/CD, unit and integration testing, automation and orchestration, including GitHub, Azure Data Factory and Azure functions.
• Familiarity with Agile workflow methodologies, demonstrating an ability to work effectively in a dynamic, iterative development environment and manage tasks with varying priorities.

What’s In It For You

• Remote Work Practices: Cogstate is a virtual first company. Cogstate employees can work from anywhere where Cogstate is registered to business within the United States, Australia, or the United Kingdom!
• Generous Paid Time-off: Cogstate employees receive 20 days of vacation leave, 10 days of personal leave and 10 paid public holidays.
• 401(k) Matching: As you invest in yourself and your future, Cogstate invests in you too: we match up to3% of your yearly salary in Cogstate’s 401k program.
• Competitive Salary: We offer competitive base salaries plus additional earning opportunities based on the position.
• Health, Dental & Vision Coverage: We've invested in comprehensive health & dental insurance options with competitive company contributions to help when you need it most. We also offer free vision insurance for all full-time employees.
• Short-Term & Long Term Disability Life Insurance: 100% employer sponsored
• Pre-Tax Benefits: Healthcare and Dependent Care Flexible Spending Accounts
• Learning & Development Opportunities: Cogstate offers a robust learning program from mentorships to assistance with programs to improve knowledge or obtain certifications in applicable areas of interest.

Our Culture
We bring our whole selves to work every day. We’re courageous and we deliver together. We’re passionate individuals who enjoy working together. We’re brave enough and care enough to have the right conversations to get the best outcome and are famous for our can-do attitude. We see challenges as opportunities and move with pace to achieve our goals.

If you’re ready to help us in our journey to optimize the measurement of brain health around the world, please apply now!

Applicants with disabilities may be entitled to reasonable accommodation under the terms of the Americans with Disabilities Act and certain state or local laws. A reasonable accommodation is a change in the way things are normally done which will ensure an equal employment opportunity without imposing undue hardship on the company. If you need assistance in applying please email PeopleandCulture@cogstate.com.

Privacy Notice for Job Applicants

Cogstate is committed to protecting your personal data. We collect and process your information for recruitment purposes in compliance with applicable laws, including the  Australian Privacy Principles (APPs), the UK General Data Protection Regulation (UK GDPR), California Consumer Privacy Act (CCPA), Virginia Consumer Data Protection Act (VCDPA), Colorado Privacy Act (CPA), and similar laws in other jurisdictions.

For more information on how we collect, use, and protect your data, and your rights under these laws, you can find Cogstate's privacy policy by clicking here.