Mathematical Scientist for AI Safety Research

Frontier AI companies are throwing billions of dollars into scaling existing architectures and methods such as next-token-prediction, direct preference optimization (DPO), reinforcement learning with human feedback (RLHF), and reinforcement learning with verified rewards (RLVR). These methods are very powerful, yet fundamentally flawed, resulting in misalignment, sycophancy, systematic biases, and other forms of harmful behavior that are already having severely negative consequences in our society.

LawZero is a non-profit founded by Yoshua Bengio developing a fundamentally new approach, the Scientist AI. We are inspired by the fact that scientific theories are both generally useful and, unlike an untrusted agent, equivariant to the consequences of their use (for a broad overview, see our blogpost). We aim not only to build a novel, safe-by-design system, but to construct a theoretical blueprint for safe AI systems, which are at once capable and come with probabilistic safety guarantees, under finite data and compute constraints.

We are seeking an outstanding theoretical researcher to contribute to the foundational and theoretical efforts here working with Yoshua Bengio and our world-class team of researchers and engineers.

 

Key responsibilities

• Boldly go where no one has gone before in exploring new AI system architectures, theoretical frameworks, and new safety mechanisms in search of a  blueprint for safe AGI and corresponding safety guarantees..
• Work closely with Yoshua Bengio and other leading researchers to analyze and refine the formal requirements needed to safely build and deploy the Scientist AI, including proving probabilistic safety guarantees under finite data and compute constraints; and, as the system matures, helping our research teams with empirical validation.
• Analyze, evaluate and contribute to the existing theoretical work already undertaken at LawZero by Yoshua Bengio and leading researchers, including the writing and dissemination of scientific papers.
• Read vigorously to stay up-to-date with relevant AI safety and machine learning research and deeply understand the risks posed by frontier models, including misalignment, reward hacking, instrumental goals, etc.
• Scrutinize and contribute to LawZero’s broader research agenda in order to guide and prioritize experiments to answer critical safety questions efficiently, in a way that does not prevent the highest level of capability.
• Share your knowledge about AI Safety and help researchers and engineers at LawZero better design and implement new ML algorithms and experiments.

 

Skills and qualifications

• A PhD  in machine learning, computer science, mathematics, statistics, or an adjacent area relevant to the Scientist AI research program.
• A track record of research, especially in the formalization and fruitful technical analysis of subtle natural-language intuition.
• Strong interest and ideally experience with technical AI safety, including an exceptional ability for abstract systems thinking.
• A deep understanding of the mathematical underpinnings commonly used in the design and analysis of probabilistic machine learning, especially for neural networks and language models. 
• The perseverance to work through hard and ill-defined problems, and to use confusion and dead-ends as challenges to dig deeper and further understanding.
• Clear communications: the ability to articulate complex ideas verbally and in writing to both technical and non-technical audiences.
• The ability to work in a collaborative environment and contribute to collective research goals.

 

Nice to have:

• The ability to regularly travel alongside Yoshua and team members to conferences is a big plus, as it will significantly accelerate the work!

 

What we offer

• The chance to contribute meaningfully to a globally critical initiative
• Comprehensive health benefits (including mental health and wellness management account)
• 20 days of vacation per year upon start
• Employer contribution of 4% to your retirement savings, with no required employee match
• Additional compensation totaling 8% of your salary to apply towards additional retirement savings or bonuses (independent of group and individual performance)
• A team of passionate world-class experts in their field
• A collaborative and inclusive work environment in our vibrant office space in the heart of Little Italy, in the trendy Mile-Ex district, close to public transportation

About LawZero

LawZero is a non-profit organization committed to advancing research and creating technical solutions that enable safe-by-design AI systems. Its scientific direction is based on new research and methods proposed by Professor Yoshua Bengio, the most cited AI researcher in the world. Based in Montreal, LawZero’s research aims to build non-agentic AI that learns primarily to understand the world rather than to act in it, giving truthful answers to questions based on transparent and externalized probabilistic reasoning. Such AI systems could be used to accelerate scientific discovery, to provide oversight for agentic AI systems, and to advance the understanding of AI risks and how to avoid them. LawZero believes that AI should be cultivated as a global public good—developed and used safely towards human flourishing. For more information, visit www.lawzero.org

You belong here

At LawZero, diversity is important to us. We value a work environment that is fair, open and respectful of differences. We welcome applications from highly qualified individuals interested in working towards our mission in a respectful, inclusive and collaborative setting.

Your personal information will be collected and processed by LawZero to evaluate your application for employment in compliance with our Privacy Policy. Under privacy laws in force in your country of residence, you may have several privacy rights, such as to request access to your personal information or to request that your personal information be rectified or erased. Details on how you can exercise your rights can be found in our Privacy Policy.