If superhuman AI would be catastrophic for humanity, why don’t we just… not invent it?
The AI Safety community has many answers to this question, some of which we’ve covered previously: natural selection prefers AIs over humans, race dynamics among powerful actors, etc. But these responses lack a flavour of necessity; AI doom scenarios seem contingent on observations about economics, politics, sociology, etc. Perhaps, with effective regulation, we can defuse the current economic and political climate such that no one wants to develop AGI.
This week Blaine Rogers, our beloved chair, will examine one promising approach to regulation that seeks to directly prevent the invention of AGI. Start by noticing that training modern foundation models like GPT-4 requires datacentresworth of specialized computer chips, like TPUs, or A100s. The supply chain for these chips is relatively easy to control; only a few companies make them. With the cooperation of these companies, we can track and monitor the use of these chips, in such a way to prevent any actor from having enough of them to train a foundation model.
As advanced machine learning systems' capabilities begin to play a significant role in geopolitics and societal order, it may become imperative that (1) governments be able to enforce rules on the development of advanced ML systems within their borders, and (2) countries be able to verify each other's compliance with potential future international agreements on advanced ML development. This work analyzes one mechanism to achieve this, by monitoring the computing hardware used for large-scale NN training. The framework's primary goal is to provide governments high confidence that no actor uses large quantities of specialized ML chips to execute a training run in violation of agreed rules. At the same time, the system does not curtail the use of consumer computing devices, and maintains the privacy and confidentiality of ML practitioners' models, data, and hyperparameters. The system consists of interventions at three stages: (1) using on-chip firmware to occasionally save snapshots of the the neural network weights stored in device memory, in a form that an inspector could later retrieve; (2) saving sufficient information about each training run to prove to inspectors the details of the training run that had resulted in the snapshotted weights; and (3) monitoring the chip supply chain to ensure that no actor can avoid discovery by amassing a large quantity of un-tracked chips. The proposed design decomposes the ML training rule verification problem into a series of narrow technical challenges, including a new variant of the Proof-of-Learning problem [Jia et al. '21].