Alignment Problem Redux

Epistemic status: Model tentatively held and personally endorsed as a good description of the generator of "alignment problem intuitions",1 although many concepts involved therein demand further scrutiny. I don't claim originality. As far as I can tell, all the ideas were present in others' writing, be it implicitly or explicitly.2 I wrote this up because I felt that an exposition like this one was missing.Suppose that you have a mind that is very generally-capable. That is, it can understand a…