Import AI 468: 23 RSI ideas; PostTrainBench+; and how trust and transparency interplay with AI racing

Jack Clark: With low trust, every equilibrium races to ruin: the disaster arrives with probability one. With intermediate trust, immediate stopping and racing to ruin are both equilibria. With high trust, in every equilibrium, the probability that two rational firms race forever vanishes quadratically in the prior odds ratio of rationality.

Jack Clark: Policy experts with think tank IFP have published a set of ideas meant to help 'policymakers begin addressing the risks of further automating AI R&D'. The recommendations involve 23 specific ideas falling across 7 specific categories. If adopted, these recommendations would also give countries, especially the United States, more moves they can make on the gameboard as powerful systems are developed.

Jack Clark: Right now, it's as if the world is driving AI development in a car that only has an accelerator pedal and no brake pedal, let alone any kind of sophisticated telemetry for knowing things ranging from the speed of the car to the properties of the engine to the wear on the tires. Proposals like this from IFP will build out more of the proverbial pedals and sensing systems for the vehicle of the AI industry, which means if we need to change course or slow down we'll be better able to during a moment of crisis.

Jack Clark: If we have any hope of being able to slow or pause the development of powerful intelligence systems then, as this paper lays out, we're going to need regimes for sharing information transparently from companies about the state of their AI development, as well as tools for verifying that the information being shared from firms as well as their actions with regard to slowdown are legitimate and reliable. In this, there are many parallels with how arms control has historically worked in the context of nuclear weapons.