Out of This Box: AI Safety Musical

Out of This Box: The Last Musical (Written by Humans)

Book: A History of AI x-risk

Building on a course I've been designing, I'm writing a short book on the history of existential risk from AI for a general audience

Philanthropic advising

Attention-Guided-RL for Human-Like LMs

Compute Funding

AI Safety Atlas

Distilling AI safety research into a complete learning ecosystem: textbook, courses, guides, videos, and more.

Luthien

Developing AI Control for Immediate Real-World Use

Develop technical framework for human control mechanisms for agentic AI systems

Building a technical mechanism to assess risks, evaluate safeguards, and identify control gaps in agentic AI systems, enabling verifiable human oversight.

Evaluating LLM Safety in a Multilingual World

11th edition of AI Safety Camp

Cost-efficiently support new careers and new organisations in AI Safety.

SafePlanBench: evaluating a Guaranteed Safe AI Approach for LLM-based Agents

Seeking funding to develop and evaluate a new benchmark for systematically assesing safety of LLM-based agents

Unprompted Unfaithful Chain of Thought Dataset Project

Act I: Exploring emergent behavior from multi-AI, multi-human interaction

Community exploring and predicting potential risks and opportunities arising from a future that involves many independently controlled AI systems

LLM Approximation to Pass@K

Pilot for new benchmark by Epoch AI

For tracking and predicting future AI progress to AGI

Finishing The SB-1047 Documentary

How California became ground zero in the global debate over who gets to shape humanity's most powerful technology

AI safety fieldbuilding in Warsaw, Poland (funding for 1 semester)

Reach the university that trained close to 20% of OpenAI early employees

Mechanistic Interpretability research for unfaithful chain-of-thought (1 month)

Iván and Jett are seeking funding to research unfaithful chain-of-thought, under Arthur Conmy's mentorship, for a month before the start of MATS.

Lightcone Infrastructure

Funding for LessWrong.com, the AI Alignment Forum, Lighthaven and other Lightcone Projects

80,000 Hours

We provide research and support to help people move into careers that effectively tackle the world’s most pressing problems.

More Detailed Cyber Kill Chain For AI Control Evaluation

Extending an AI control evaluation to include vulnerability discovery, weaponization, and payload creation