Back to jobs

Senior Data Scientist (Psychometrics)

Remote, DC

About Us

At Udemy, we’re on a mission to improve lives through the power of learning. We’re a leading global learning company and one of the world’s largest education platforms, with more than 67 million learners. Our goal is to provide flexible, effective skill development to empower organizations and individuals. 

Talented people are everywhere, and the right opportunity can be hard to come by. That’s why we’re focused on revolutionizing learning, using our skills and expertise to help others develop theirs and reach their full potential. Individually, we bring our unique perspective to reimagine the way we share knowledge. Together, we can improve lives by making learning more accessible for our learners, our instructors, and businesses around the world.

About you 

You’re an analytical problem-solver ready to put your skills toward purposeful work that has a global impact. You want to lead the way in innovation, exploring the latest technologies and finding new solutions. You thrive in a collaborative environment and are eager to work with and learn alongside the best in Product, Design, and Engineering.

About this role 

In the Data organization at Udemy, we’re passionate about transforming the future of education using data!  Psychometrics plays an important role in Udemy’s growing skills validation strategy. We’re looking for fun, collaborative, self-motivated psychometricians with an insatiable sense of curiosity and a knack for asking the right questions.

Our ideal candidate is an analytically creative thinker who is unafraid of ambiguity, has a track record of tackling difficult technical challenges, and has a pragmatic and relentless focus on driving meaningful business impact. 

We believe in building diverse teams, collaborative teams in which each person brings unique skills, knowledge, and experience to the table. If this opportunity sounds exciting to you, then read on!

As a psychometrician at Udemy, you will be partnering with product managers, designers, and engineers to build algorithmic and model-driven solutions to power skill assessments. This role will require a combination of applied measurement experience, data-driven decision making, and the data science and coding chops to dive headfirst into development on production systems. A successful candidate will have knowledge of best industry practices for growing and maintaining a large set of assessment products, while being courageously experimental in finding new solutions to ongoing issues in test development and maintenance.

What you'll be doing

  • Collaborate with other psychometricians, data scientists, PMs, designers, and engineers to maintain and grow our skill assessment platform
  • Effectively communicate and present to both technical and non-technical audiences
  • Critically evaluate and implement the state-of-the-art research literature in educational assessment
  • Develop production-grade data pipelines and continuously improve the quality and accessibility of data
  • Develop psychometric models for skill assessment through all phases, including prototyping, productionization, and experimental evaluation
  • Define metrics and build dashboards to track the adoption and health of skill assessments
  • Collaborate with content developers and instructional designers to develop and refine item writing and review processes

What you’ll have

  • 3+ years of hands-on industry experience in a psychometric or data science-oriented role
  • Domain experience in educational assessment and psychometrics (e.g., Classical Test Theory (CTT), Item Response Theory (IRT), Computerized Adaptive Testing (CAT), equating and scaling) 
  • Experience creating or applying novel measurement models (Diagnostic Classification, Knowledge Tracing, MIRT) and their associated estimation algorithms
  • Strong SQL ability and proficiency with 1+ programming languages (e.g., R or Python)
  • End-to-end experience organizing and analyzing large data sets for operational purposes
  • Ability to communicate effectively with non-technical stakeholders
  • Strong knowledge of applied statistics and experimental design
  • Ability to work autonomously, relentlessly prioritize, and collaborate with team members across time zones

We understand that not everyone will match each of the above qualifications. However, we also realize that everyone has unique experiences that can add value to our company. Even if you think your background might not perfectly align, we'd love to hear from you!

We Are More Excited About You Because You Have:

  • 5+ years of relevant industry experience
  • Experience with machine learning and causal inference
  • Familiarity with standard software engineering practices (e.g., Scrum)
  • Experience using big data technologies (e.g., Hadoop, Spark)

Life at Udemy

We aspire to be as vibrant and dynamic as the communities we serve, as inquisitive as those who use our platform, and as revolutionary as the future we strive to open for everyone. Here are some of the things we love about life at Udemy:

We’re invested in creating an inclusive environment that welcomes a diverse range of backgrounds and experiences. From creating employee resource groups, ensuring we’re a Fair Pay Workplace, and building a flexible work culture, our belonging, equity, diversity, and inclusion (BEDI) initiatives always put our people first. We want you to be able to bring your authentic self to work because when we all do, we’re better for it.

Learning is what we do – inside and out. Our Learning & Development team is second to none, helping ensure your journey is one of continuous progression. You’ll also have unlimited access to Udemy courses, monthly UDays (meeting-free professional development days), and a generous annual professional development stipend.

Our reason to exist is to revolutionize learning – that calls for taking risks and learning from failures. Whether it’s our hackathons (a company-wide effort to envision new possibilities for our product) or sharing our prototypes, we see experimentation as a crucial step on the path to success.

We’re committed to creating world-class employee experiences and are proud of the recognition of this by Great Place to Work. 

Of course, the best thing about being part of Udemy is knowing your work makes a difference for people and organizations around the world. You’ve got the skills; why not use them to help others develop theirs?

At Udemy, we value diversity and inclusion and consider qualified applicants without regard to race, color, religion, sex, national origin, ancestry, age, genetic information, sexual orientation, gender identity, marital or family status, veteran status, medical condition, or disability. We will consider for employment qualified applicants with arrest and conviction records.

Our Benefits Start With U

Our benefits package at Udemy “starts with U” and is grounded in mission-driven benefits that intentionally align with our core values. Here’s a sneak peek at just a few highlighted benefits if you’re a full-time Udemate based in the United States:

Core Benefits: Comprehensive medical, dental, vision, life/AD&D, and disability coverage

HSA/FSA/Commuter: Pre-tax savings/spending plans, including HSA employer contributions

Transgender Benefits: Expanded medical coverage for gender-affirming care

Parental Benefits: Financial assistance for fertility (including egg-freezing), adoption, and surrogacy expenses; twelve weeks of paid family bonding for birth, surrogacy, adoption, and foster placement (plus up to six to ten weeks of paid disability leave for birthing parents) as well as a two-week “ease-back” program that enables parents to return part-time at full pay

Whole U: Fully paid memberships for meditation/mindfulness, therapy/coaching visits, financial planning, fertility and family building, maternity and newborn care, parenting and pediatrics, menopause care, breastmilk shipping, and caregiving support

Learning: Free access to Udemy and Udemy Business plus a yearly stipend to spend on professional development

Retirement: Pre-tax and Roth 401(k) plan with an employer contribution

Vacation: Flexible vacation for salaried Udemates and fifteen days per year for hourly

Holidays: Twelve paid holidays throughout the year

Information regarding data privacy is available within the Udemy Careers Privacy Notice.

Apply for this job

*

indicates a required field

Resume/CV

Accepted file types: pdf, doc, docx, txt, rtf

Cover Letter

Accepted file types: pdf, doc, docx, txt, rtf

Select...
Select...

We want to understand all of the ways that you have interacted or been exposed to Udemy so that we can continue to invest in efforts that resonate with candidates.

Select...
Select...

Saying “No” to this question indicates you are eligible for work in the United States and do not require sponsorship.

Select...

Voluntary Self-Identification

For government reporting purposes, we ask candidates to respond to the below self-identification survey. Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiring process or thereafter. Any information that you do provide will be recorded and maintained in a confidential file.

As set forth in BEDI Partnerships’s Equal Employment Opportunity policy, we do not discriminate on the basis of any protected group status under any applicable law.

Select...
Select...
Race & Ethnicity Definitions

If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection. As a government contractor subject to the Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measure the effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categories is as follows:

A "disabled veteran" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability.

A "recently separated veteran" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.

An "active duty wartime or campaign badge veteran" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.

An "Armed forces service medal veteran" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985.

Select...

Voluntary Self-Identification of Disability

Form CC-305
Page 1 of 1
OMB Control Number 1250-0005
Expires 04/30/2026

Why are you being asked to complete this form?

We are a federal contractor or subcontractor. The law requires us to provide equal employment opportunity to qualified people with disabilities. We have a goal of having at least 7% of our workers as people with disabilities. The law says we must measure our progress towards this goal. To do this, we must ask applicants and employees if they have a disability or have ever had one. People can become disabled, so we need to ask this question at least every five years.

Completing this form is voluntary, and we hope that you will choose to do so. Your answer is confidential. No one who makes hiring decisions will see it. Your decision to complete the form and your answer will not harm you in any way. If you want to learn more about the law or this form, visit the U.S. Department of Labor’s Office of Federal Contract Compliance Programs (OFCCP) website at www.dol.gov/ofccp.

How do you know if you have a disability?

A disability is a condition that substantially limits one or more of your “major life activities.” If you have or have ever had such a condition, you are a person with a disability. Disabilities include, but are not limited to:

  • Alcohol or other substance use disorder (not currently using drugs illegally)
  • Autoimmune disorder, for example, lupus, fibromyalgia, rheumatoid arthritis, HIV/AIDS
  • Blind or low vision
  • Cancer (past or present)
  • Cardiovascular or heart disease
  • Celiac disease
  • Cerebral palsy
  • Deaf or serious difficulty hearing
  • Diabetes
  • Disfigurement, for example, disfigurement caused by burns, wounds, accidents, or congenital disorders
  • Epilepsy or other seizure disorder
  • Gastrointestinal disorders, for example, Crohn's Disease, irritable bowel syndrome
  • Intellectual or developmental disability
  • Mental health conditions, for example, depression, bipolar disorder, anxiety disorder, schizophrenia, PTSD
  • Missing limbs or partially missing limbs
  • Mobility impairment, benefiting from the use of a wheelchair, scooter, walker, leg brace(s) and/or other supports
  • Nervous system condition, for example, migraine headaches, Parkinson’s disease, multiple sclerosis (MS)
  • Neurodivergence, for example, attention-deficit/hyperactivity disorder (ADHD), autism spectrum disorder, dyslexia, dyspraxia, other learning disabilities
  • Partial or complete paralysis (any cause)
  • Pulmonary or respiratory conditions, for example, tuberculosis, asthma, emphysema
  • Short stature (dwarfism)
  • Traumatic brain injury
Select...

PUBLIC BURDEN STATEMENT: According to the Paperwork Reduction Act of 1995 no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. This survey should take about 5 minutes to complete.