Staff Applied AI Engineer (Observability)
We Breathe Life Into Data
At Komodo Health, our mission is to reduce the global burden of disease. And we believe that smarter use of data is essential to this mission. That’s why we built the Healthcare Map — the industry’s largest, most complete, precise view of the U.S. healthcare system — by combining de-identified, real-world patient data with innovative algorithms and decades of clinical experience. The Healthcare Map serves as our foundation for a powerful suite of software applications, helping us answer healthcare’s most complex questions for our partners. Across the healthcare ecosystem, we’re helping our clients unlock critical insights to track detailed patient behaviors and treatment patterns, identify gaps in care, address unmet patient needs, and reduce the global burden of disease.
As we pursue these goals, it remains essential to us that we stay grounded in our values: be awesome, seek growth, deliver “wow,” and enjoy the ride. At Komodo, you will be joining a team of ambitious, supportive Dragons with diverse backgrounds but a shared passion to deliver on our mission to reduce the burden of disease — and enjoy the journey along the way.
The Opportunity at Komodo Health:
Healthcare in the U.S. is fragmented and inefficient. Komodo Health is working to fix that—with data.
By building the most complete view of patient journeys across the U.S., Komodo enables life sciences companies, payers, and providers to make better decisions that directly improve patient outcomes.To unlock the full potential of this data, Komodo is investing in AI-native infrastructure—systems that make AI reliable, scalable, and deeply embedded into how products are built and used.
Mission of the Role:
We’re hiring Staff Applied AI Engineers to join a newly formed team focused on building the foundation that makes AI systems trustworthy at scale. This is a greenfield, backend-heavy systems role where you’ll define and build the core infrastructure that powers AI across the company. Your entry point is AI observability, evaluation, and reliability—but that is not the destination.
You’ll start by building the systems that make AI measurable and debuggable. Then you’ll expand into broader platform ownership: agent architectures, orchestration systems, and company-wide AI standards.
Looking back on your first 12 months at Komodo Health, you will have accomplished…
- Establish the observability foundation for AI systems across Komodo (logging, tracing, evaluation pipelines, feedback loops)
- Define how the organization measures LLM performance and reliability (failure modes, hallucinations, drift, system degradation)
- Build reusable infrastructure and frameworks adopted across multiple teams
- Set standards for AI system quality, monitoring, and production readiness
- Influence technical direction and platform decisions across the AI org
- Expand into broader AI platform ownership (multi-agent systems, orchestration layers, shared services)
- Mentor engineers in AI system design, debugging, and evaluation
What You’ll Own
AI Observability & Reliability Foundation (Initial Focus)
- Design and build systems for:
- Logging, tracing, and request visibility
- Evaluation pipelines and benchmarking frameworks
- Feedback loops for continuous system improvement
- Develop approaches to detect hallucinations, model drift, performance regressions and system failures
- Establish organization-wide standards for AI monitoring and evaluation
AI Platform Expansion (Ongoing Scope)
- Build foundational AI systems: agent frameworks, orchestration and tool-calling systems, and shared infrastructure for LLM-powered applications
- Contribute to company-wide AI architecture and standards
- Make high-impact build vs. buy decisions for AI tooling and platforms
- Partner cross-functionally to ensure systems are scalable, reliable, secure and production-ready
What you bring to Komodo Health (required):
- Strong experience building production AI systems (not just prototypes)
- Deep expertise in LLMs and applied AI systems: tool/function calling, agent orchestration and prompt systems
- Hands-on experience with:
- AI observability, evaluation, or reliability systems
- Monitoring, debugging, and performance analysis in production
- Strong backend engineering skills: Python, APIs, distributed systems, or platform architecture
- Proven ability to operate in ambiguous, undefined environments
- Track record of owning systems end-to-end and driving technical direction
- You will operate as a strategic AI leader—setting patterns, driving adoption, and ensuring Komodo stays at the forefront of applied AI innovation.
Additional skills and experience we’d prioritize (nice to have)…
- Experience building internal observability platforms and valuation frameworks for LLM systems.
- Familiarity with: Request tracing, replay systems, or model monitoring
- Experience leading large-scale distributed data and compute systems (e.g., Spark, Databricks, Snowflake).
- Healthcare data experience (not required)
** Location flexible to NYC or SF hybrid, and remote
The pay range for each job posting reflects a minimum and maximum range of annual base pay that we reasonably expect to pay for this position within the US. We carefully consider multiple business-related factors when determining compensation, including job-related skills, work experience, geographic work location, relevant training and certifications, business needs and market demands.
The starting annual base pay for this role is listed below. This position may be eligible for performance-based bonuses as determined in the Company’s sole discretion and in accordance with a written agreement or plan. This role may also be eligible for equity awards. In addition, this role is eligible for benefits including, but not limited to, comprehensive health, dental, and vision insurance; flexible time off and holidays; 401(k) with company match; disability insurance and life insurance; and leaves of absence in accordance with applicable state and local laws and regulations and company policy.
San Francisco Bay Area and New York City:
$274,000 - $322,000 USD
All Other US Locations:
$238,000 - $280,000 USD
Komodo's AI Standard
At Komodo, we're not just witnessing the AI revolution – we're leading it. This is a pivotal moment in time, where being first to market with AI transforms industries and sets the bar. We've already established industry leadership in leveraging AI to revolutionize healthcare, and we expect every team member to contribute. AI here isn't optional; it's foundational. We expect you to integrate AI into your daily work – from summarizing documents to automating workflows and uncovering insights. This isn't just about efficiency; it's about making every moment more meaningful, building on trust in AI, and driving our collective success.
Join us in shaping the future of healthcare intelligence.
Where You’ll Work
Komodo Health has a hybrid work model with hubs in San Francisco, New York City, and Chicago. Roles vary — some can be performed from anywhere in the country, others are scoped to a specific region, and some are based near one of our hubs. For hub-based Dragons, we're building intentional in-office rhythms alongside the flexibility that's core to how we work. Whatever your setup, expectations will always be clear before you join.
Equal Opportunity Statement
Komodo Health provides equal employment opportunities to all applicants and employees. We prohibit discrimination and harassment of any type with regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws.
By submitting your application, you acknowledge that you have read and understand Komodo Health’s Privacy Notice for Employees and Contractors.
This notice explains how we collect, use, and retain applicant data.
Create a Job Alert
Interested in building your career at Komodo Health? Get future opportunities sent straight to your email.
Apply for this job
*
indicates a required field