"To play a wrong note is insignificant; to play without passion is inexcusable."
Hyperalignment
Aligning intelligent, agentic systems to human values is misguided, and we ought instead focus on incorporating the rich computational structure of metaphysical objects, based on insights from moral epistemology.
Strategic Deceleration
However, incremental progress towards hyperalignment would be of limited use if civilization disempowerment occurs in the meantime. This line of work is focused on buying time under a first-principles threat model.
Strategic Acceleration
Besides, hyperalignment work prior to automated research will likely amount to little, given future millenia of intellectual progress in a glimpse. This line of work is focused on capturing value from capability advances through "agent readiness" efforts.
Acknowledgements
I'm grateful for having been supported in my work, either financially or through compute, either recently or less so, by Open Philantropy, Long-Term Future Fund, Google Research, Conjecture, Stability AI, AI Safety Camp, and Andreas Stuhlmüller.