Medical Scientific Annotation Specialist

Overview

Black Canyon Consulting (BCC) is seeking a highly qualified Medical Scientific Annotation Specialist to support a critical NIH initiative focused on advancing clinical research through the structured curation of case reports for type 2 diabetes. This role involves validating and refining outputs from large language models (LLMs) that extract clinical and longitudinal data from unstructured medical case reports. Ideal candidates will have a strong clinical background—preferably in diabetes care—and prior experience with case report interpretation or medical text annotation.

 Purpose

The primary aim of this project is to improve the accuracy and utility of LLM-derived outputs by annotating phenotype data, treatment events, and high-risk disease progression periods in case reports related to type 2 diabetes and GLP-1RA treatments.

Roles & Responsibilities

• Training & Onboarding

  • Participate in remote training sessions to familiarize with:

    • Annotation guidelines and protocols provided by NIH.

    • Tools for reviewing LLM-extracted timelines and clinical events.

  Annotation Tasks

  • Phenotyping & Treatment Validation

    • Review LLM-generated annotations for diagnosis, treatment, and outcomes.

    • Validate and correct phenotype classifications and therapy identification (e.g., GLP-1RA use).

  • Time-Series & Event Verification

    • Assess the accuracy of clinical events (e.g., symptom onset, treatment changes, lab results) and their placement within a patient timeline.

    • Correct any errors in event timing or sequencing.

  • High-Risk Period Assessment

    • Identify periods of heightened risk (e.g., HbA1c spikes, treatment failures).

    • Confirm alignment with supporting evidence (labs, symptoms, interventions) in the case report.

  Quality Control & Review

  • Double-review each annotated case report in collaboration with other team members.

  • Achieve and maintain inter-annotator agreement above defined thresholds.

  • Participate in discussions to resolve inconsistencies or ambiguities.

  Communication

  • Engage in regular feedback meetings to discuss project updates, challenges, and improvements to annotation guidelines.

 

Required Skills

• Clinical training and familiarity with interpreting medical case reports.

• Knowledge of type 2 diabetes management and GLP-1RA therapies.

• Excellent attention to detail and consistency in data annotation.

• Comfort working with digital annotation tools and structured forms.

• Ability to work both independently and collaboratively in a distributed team setting.

 Education & Experience

• Required:

  • Advanced degree or certification in a clinical field (e.g., PA, NP, MD, RN with clinical experience).

  • Experience reading, interpreting, or writing case reports.

  Preferred:

  • Specialization or significant experience in endocrinology, diabetes treatment, or internal medicine.

  • Prior exposure to annotation projects, natural language processing (NLP), or LLM output review.

Work Environment & Schedule

• Location: Remote (after training), with initial virtual training provided by NIH.

• Schedule: Flexible hours, but availability required during U.S. Eastern business hours for meetings.

• Project Duration: Estimated 4–5 weeks per annotator, with potential extensions.

Deliverables

• Review and curation of LLM-based predictions

• Accurate validation of timelines, phenotypes, and high-risk periods across annotated documents.

• Adherence to NIH-supplied annotation protocols and documentation standards.

Apply Today!
Interested candidates are encouraged to apply through our website. Please submit your resume and any relevant documentation showcasing experience in biomedical annotation or related fields.