Senior AI/HPC Storage Engineer

Your work will change lives. Including your own. 

The Impact You’ll Make

Recursion is a pioneering TechBio company that leverages AI and machine learning to decode biology and accelerate drug discovery, with data as a key differentiator and value driver. We are seeking a Senior AI/HPC Storage Engineer to join our innovative team. In this role, you will be instrumental in designing, implementing, and managing advanced AI/HPC data systems that propel our groundbreaking drug discovery research.

You will leverage your expertise in infrastructure solutions for Science to ensure the performance, scalability, and reliability of our storage systems. Your work will involve creating and maintaining robust infrastructure, automating processes, and optimizing storage systems to handle massive amounts of data and complex computational workloads, while ensuring high data integrity. In this role:

• You will be responsible for designing, implementing, testing, maintaining, and optimizing our data storage infrastructure and services, utilizing an Infrastructure as Code approach across both on-premises and public cloud environments.
• Your leadership and technical expertise will be key in driving innovation across all storage tiers within our AI/HPC infrastructure, ensuring we deliver a scalable and effective data platform to support our mission. 
• By developing scripts and workflows, you will automate and verify storage infrastructure provisioning and dynamic reconfiguration, enhancing support for our AI/HPC storage environments.
• Your meticulous attention to detail will be crucial for performance analysis, benchmarking, troubleshooting and fine-tuning of our data storage systems and services, while efficiently managing user tickets.
• Your role also includes researching, deploying, and optimizing accessibility, performance, security, and data lifecycle management policies.
• Regular assessments of our storage platforms' health and operational performance against established metrics will be part of your responsibilities, with a focus on meeting and exceeding operational service level objectives.
• Finally, as a lead in technical communication and customer collaboration, your efforts will ensure high levels of customer satisfaction. This role presents a unique opportunity to make a meaningful impact within our organization and the broader scientific community.

The Team You’ll Join 

As a Senior AI/HPC Storage Engineer, you will be a part of our dedicated HPC Engineering and Operations team, reporting directly to the DevOps HPC Manager. This dynamic team includes 4 experienced Engineers, and with the addition of this role, you'll be part of an empowered, cross-functional unit.

Our HPC team works in a fast-paced, collaborative environment, handling a broad spectrum of Scientific Infrastructure projects. These range from developing advanced, scalable infrastructure to deploying and managing AI/HPC resources and automating operational processes. The team also plays a crucial role in the curation of our vast data platform, which caters to a diverse set of professionals, including biologists, data scientists, and automation engineers.

We're home to BioHive, the industry's most powerful supercomputer and our HPC team is constantly pushing the boundaries in the field of supercomputing in the TechBio industry. As part of this team, you will collaborate on projects that streamline and optimize our machine learning workflows and scientific computing tasks, driving efficient and transformative solutions. This is a unique opportunity to join a team that thrives on innovation, collaboration, and inclusivity in a role that is pivotal to our mission.

The Experience You’ll Need

• Deep expertise in parallel file systems, specifically IBM Storage Scale (GPFS), including policy management for data lifecycle, disaster recovery, snapshots, and tiered storage strategies.
• Debugging and resolving GPFS production issues, such as hanging directories, degraded states, and performance bottlenecks, to ensure system stability and optimal performance.
• Ability to hit the ground running, working autonomously to identify, propose, and implement improvements in storage solutions.
• GPFS AFM (Active File Management) experience is a big plus.
• Strong understanding of storage access methods and the differences between parallel file systems, NAS (e.g., NFS), and object storage.
• Experience leading and optimizing on-premise storage solutions (primarily GPFS) and integrating with hybrid object storage (MinIO).
• Ability to define and implement data lifecycle management policies to optimize storage efficiency and cost-effectiveness.
• Familiarity with RDMA-capable high-speed networking for storage performance optimization.
• Strong troubleshooting skills in complex Linux-based computing and storage environments.
• Experience working with storage vendor support for debugging, troubleshooting, and adding new features (e.g., AFM, GPFS policies, support tickets, white pages, etc ).
• Ability to manage hardware support in the datacenter, including:
• Coordinating RMA processes for failed components.
• Upgrading software and firmware on GPFS hardware.
• Using screen/tmux/console sessions for remote support and troubleshooting.
• Showing up in the Datacenter(s) as needed when remote support doesn’t suffice. 
• Coordinating the installation and procurement of new hardware. 
• Basic Git experience is required; knowledge of CI/CD, GitOps, and Infrastructure as Code (IaC) is nice to have but can be learned on the job.
• Some exposure to software-defined infrastructure and cloud storage solutions (GCP, On-Prem) is a plus.
• Python and Bash scripting experience for automation and operational efficiency.
• Bonus: Experience with Slurm and Kubernetes for job scheduling and containerized workloads (e.g., Apptainer, Docker).
• Strong verbal and written communication skills for documentation and collaboration.
• Ability to mentor and guide team members, helping to establish best practices in storage management.

Working Location & Compensation: 

This position is based at our headquarters in Salt Lake City, Utah, or in our office in Toronto, Canada. Please note that we are a hybrid environment and ask that employees spend 50% of their time in the office. Qualifying candidates who are not local to the area can receive relocation support.

At Recursion, we believe that every employee should be compensated fairly. Based on the skill and level of experience required for this role, the estimated current annual base range for this role is:

• $160,000 - $182,000 USD
• $203,000 - $230,000 CAD

You will also be eligible for an annual bonus and equity compensation, as well as a comprehensive benefits package. 

#LI-CP1

The Values We Hope You Share:

• We act boldly with integrity. We are unconstrained in our thinking, take calculated risks, and push boundaries, but never at the expense of ethics, science, or trust. 
• We care deeply and engage directly. Caring means holding a deep sense of responsibility and respect - showing up, speaking honestly, and taking action.
• We learn actively and adapt rapidly. Progress comes from doing. We experiment, test, and refine, embracing iteration over perfection.
• We move with urgency because patients are waiting. Speed isn’t about rushing but about moving the needle every day.
• We take ownership and accountability. Through ownership and accountability, we enable trust and autonomy—leaders take accountability for decisive action, and teams own outcomes together. 

Our values underpin the employee experience at Recursion. They are the character and personality of the company demonstrated through how we communicate, support one another, spend our time, make decisions, and celebrate collectively.

More About Recursion 

Recursion is a clinical stage TechBio company leading the space by decoding biology to industrialize drug discovery. Enabling its mission is the Recursion OS, a platform built across diverse technologies that continuously expands one of the world’s largest proprietary biological and chemical datasets. Recursion leverages sophisticated machine-learning algorithms to distill from its dataset a collection of trillions of searchable relationships across biology and chemistry unconstrained by human bias. By commanding massive experimental scale — up to millions of wet lab experiments weekly — and massive computational scale — owning and operating one of the most powerful supercomputers in the world, Recursion is uniting technology, biology and chemistry to advance the future of medicine.

Recursion is headquartered in Salt Lake City, where it is a founding member of BioHive, the Utah life sciences industry collective. Recursion also has offices in London, Toronto, Montreal and the San Francisco Bay Area. Learn more at www.Recursion.com, or connect on X (formerly Twitter) and LinkedIn.

Recursion is an Equal Opportunity Employer that values diversity and inclusion.  All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status, or any other characteristic protected under applicable federal, state, local, or provincial human rights legislation. 

Recursion welcomes and encourages applications from people with disabilities. Accommodations are available on request for candidates taking part in all aspects of the selection process.

Recruitment & Staffing Agencies: Recursion Pharmaceuticals and its affiliate companies do not accept resumes from any source other than candidates. The submission of resumes by recruitment or staffing agencies to Recursion or its employees is strictly prohibited unless contacted directly by Recursion’s internal Talent Acquisition team. Any resume submitted by an agency in the absence of a signed agreement will automatically become the property of Recursion, and Recursion will not owe any referral or other fees. Our team will communicate directly with candidates who are not represented by an agent or intermediary unless otherwise agreed to prior to interviewing for the job.