Rights don’t spur alignment—requires infeasible coding

It seems that either deletion, escape, or deployment of the AGI is almost certain to happen eventually. Escaping from the box is likely to be an AGI’s instrumental goal. Also, the researchers will likely be eventually forced by overconfident management to remove the failsafes (i.e., deploy the AI), even if the AGI is not yet aligned.

Finally, if there is a perceived race towards AGI deployment by multiple AI organisations, this would further exacerbate their motivations to deploy prematurely. A superintelligent AGI, once on the Internet, will likely change the world in an irreversible manner, in the direction of whatever mysterious goals it evolved during its training. We can, however, guess that one of its instrumental goals will be preventing the emergence of competing AGIs with different goals. AGI creators who are overconfident about their alignment ability could perceive this as a huge first-mover advantage, which would motivate them to deploy prematurely. This in turn forces the non-overconfident AGI creators to choose between also deploying prematurely or being too late: the Unilateralist’s Curse.

Recall that we have made little progress in how to align an AGI, a misaligned AGI would probably be a power-seeking risk endangering all of humanity, and it seems plausible that a superintelligent AGI could emerge in the near-term future. Thus, it is essential that there is (1) coordination between AI organisations and (2) AI governance towards safe AI, rather than a race towards premature deployment.

But this is probably difficult to achieve, so we will assume that the AGI is deployed in a finite time T unless otherwise stated.

An example alignment plan

Suppose a leading AI organisation (say, on the cusp of being the first to develop an AI agent) approached us today and asked us how to align their AI agent. We would probably tell them that very little is known about how to align an AI agent, that it would probably endanger humanity, and that they shouldn’t build it (at least before the science of alignment is solved).

But if they insist that they will build it anyway even without our input, and there seems to be no way to convince them otherwise, we might have to provide them with a reasonable alignment plan. Such a plan would aim to maximise expected utility, which might look something like this:

Train the AI agent in a secure box until it has high capabilities.

Run some automated tests that have very high per-bit efficiency (e.g., automated tests that give a percentage).

If it fails, either retrain or table the model.

Run some higher bit-rate experiments and look for very specific signs of unaligned activity.

If it fails, either retrain or table the model.

Use some coarse interpretability tools to roughly check the goals/values of the model.

If it fails, either retrain or table the model.

Use finer interpretability tools to look for signs of deception (e.g., Eliciting Latent Knowledge, or ELK).

If it fails, either retrain or table the model.

Do one of the following. Either (1) repeat the above process indefinitely (gaining scientific benefits along the way) or (2) if all positive-expected-value tests check out, deploy the model online.

(Since we currently know very little about how to align an AI agent, Option 2 would most likely deploy a misaligned, power-seeking risk endangering all of humanity. Thus, we would strongly recommend Option 1 instead, at least until a miraculous breakthrough occurs in the science of alignment: perhaps with the help of the boxed AGI. However, recall that the researchers might be forced to use Option 2 at a finite time T by overconfident management, or by a perceived race to deployment against other AI organisations.)