Paul Bricman

 

Hyperalignment

Aligning intelligent, agentic systems to human values is misguided, and we ought instead focus on incorporating the rich computational structure of metaphysical objects, based on insights from moral epistemology.

Strategic Deceleration

However, incremental progress towards hyperalignment would be of limited use if civilization disempowerment occurs in the meantime. This line of work is focused on buying time under a first-principles threat model.

Strategic Acceleration

Besides, hyperalignment work prior to automated research will likely amount to little, given future millenia of intellectual progress in a glimpse. This line of work is focused on capturing value from capability advances.

Acknowledgements

I'm grateful for having been supported in my work, either financially or through compute, either recently or less so, by Open Philantropy, Long-Term Future Fund, Google Research, Conjecture, Stability AI, AI Safety Camp, and Andreas Stuhlmüller.