Nuclear Gandhi

 

 

Nuclear Gandhi is an Internet meme and urban legend relating to the 1991 video game Civilization, in which there was reportedly a bug that would eventually force the pacifist leader Mahatma Gandhi to become extremely aggressive and make heavy use of nuclear weapons. In thinking of AI alignment and in trying to program and create a benevolent artificial super-intelligence this is a more comical example of where in attempting to optimize for a certain property or characteristic that might seem maximally beneficial to humans, it ends up with unforeseen properties that end up in disastrous consequences. (its also an example of how you should always check the consequences of overflow errors for unsigned integers)

 The bug essentially goes as follows: each leader's game AI in Civilization had a parameter that described their aggression on a scale from 1 to 10, with 1 being least aggressive and 10 most aggressive.  Indian leader Mahatma Gandhi was the only leader in the game with the lowest possible aggression rating of 1, meaning they could only wage defensive wars with other players. When democracy is unlocked, which is the preferred form of government for peaceful nations, India would switch to this government in the game and in doing so follow the games rules which would decrease its aggression level by 2. 

 

Since Gandhi is already beginning the game with an aggression level of 1, when it inevitably takes on the government type of democracy, it decreases the aggression level to -1. However, the aggression level was stored as an 8-bit unsigned integer variable that could only store values in the range from 0 to 255 (or 2^8 − 1). The negative value would therefore result in an integer overflow (specifically an integer underflow), with the value being stored as 255, making him 25 times more aggressive than the most aggressive civilization in the game. Unlocking democracy also happens to be around the same time when civilizations in the game also begin to develop nuclear weapons, and so the result of this bug is a super peaceful Gandhi until the endgame when suddenly he begins cleansing the entire game map in nuclear fire posthaste. 

 

I'm reminded also of the paperclip problem in thinking about artificial intelligence. The paperclip problem is lets say that we optimize an artificial intelligence to be maximally good at doing one thing: creating paperclips. In a flash, its figured out the best possible way to create the maximum amount of paperclips, namely converting the entire earth and all its material resources into paperclips, including humans. I am also reminded of the corollary to modern capitalism, in which we live in a system that is designed to maximally good at doing one thing: generating capital. This includes destroying our entire environment in order to do this.  

 In a sense I kind of hope the development of artificial intelligence is an emergent phenomenon out of control of humans, this is somewhat terrifying perhaps but in imposing our own ideas on it there is a large chance of us fucking it up tremendously given what we think we know about intelligence and consciousness. If its emergent then it is out of our control and we just have to hope for the best, but if we have to actively decide what properties we are going to optimize for then we run into the problem of introducing errors into what we want it to do that have unintended consequences that could prove disastrous. 

Maybe we try to maximize for generating the most amount of happiness for the most amount of humans, and it ends up in a strange recursive loop in which this is somehow maximized within its model by generating the most amount of misery for humans. Some humans are masochistic, this might work out for them, and one could maybe even argue that there's some fucked up part of our average psychology that is drawn to misery and pain that we tend to repress. In some models of psychology, instead of repressing the darker or shadow aspects of ourselves we instead learn to integrate it within our conscious mind such that they both flow together and there is harmony between them. 

 Specialization is a dangerous game to play when it comes to artificial intelligence. its important to think about how to integrate certain aspects into any model such that we don't end up blowing ourselves up. Perhaps on the other end AI is actually emergent, in which case maybe all we can do is hope for a good outcome.