User blog:Ultra64Detective2401/Intermission:Personalization and AI Safety

Sequence:The Basics of Anomalies

Previous:The Process of Personalization

Next:The Search For More Anomalies, Part 1

Welcome to the first intermission of this series! Intermissions like these appear when we appear to go off on more of a tangent from the main goal of searching for anomalies within the game, and instead speculate on what’s been happening behind the scenes...

This time, we’ll be dealing with AI safety, and the hypothesis that all the anomalies of the game that weren’t what the player wanted, or fun for the player to experience, was a result of a failure on Nintendo’s part in properly implementing AI safety.

So, what is AI safety, and why would it be hard to implement?

AI safety is actually quite simple:It is the problem of getting an AI to act “friendly”:Or, more specifically, only do what it was intended to do. That’s harder than it sounds, though. And to demonstrate this, let us go through a thought experiment.

Suppose we want to build an AI with the mundane task of creating paperclips. So we design an AI with that goal in mind. The AI will start making paperclips.. upgrade its hardware to make more paperclips.. good so far... and then realize life has atoms that can be used to make paperclips with. And thus all life will soon be turned to paperclips... that’s not a good outcome at all! (In fact, the whole planet would eventually be turned into paperclips!) From the AI’s perspective, though, this is good:Now there’s more paperclips!

Just stating your goal isn’t enough. You need some way of stopping the AI from, in the process, doing things you wouldn’t want it to do. The AI in Super Mario 64 was made in the 90’s.. with the goal of personalizing copies to fit the player’s playstyle, and potentially desires. Could negative anomalies be a result of a failure of properly implementing AI safety?

Well... some of them might. But only if these kinds of anomalies could concievably lead to better personalization later on. An AI without proper AI safety (note that AI safety appears to be unsolved to this day) would definitely try to experiment on you to learn more about you, so that it can better personalize the game. After all, that is the AI’s entire goal:Anything that helps it personalize the game is something it does. Just as a paperclipper AI doesn’t care that it has killed off all life, this AI wouldn’t care about any of the harm it has caused through its experiments... unless that means its personalization becomes less effective.

But... there are a few reasons to think this can’t be the entire story.

For one, while a failed experiment like this would be a good reason for Nintendo to remove the AI from all future releases... it wouldn’t be a reason to deny the AI ever existed, censor the Oman Archive leaks, and... other things (we’ll get to those later on:for now, those two things are enough. And we know the Oman Archive leaks were censored because they didn’t contain every texture that was used [some were only found very recently], and didn’t contain any evidence for an AI.) We know this because of their response to two other failed experiments:The Virtual Boy, and the 38th episode of the Pokémon anime.

The Virtual Boy was a failure of a console. Not only that, it caused eye damage to younger players.. you know, the kinds of players the console was designed for? (We know the latter part because of how short the VB was) None of the games were that good either. Nintendo’s response to this ordeal? Discontinue the Virtual Boy, never re-release the games again, and to not try 3D again for 15 years. But, they did not deny the existence of the Virtual Boy, let alone go to this extent to hide its existence.

But okay, the Virtual Boy was a physical console, it’d be pretty hard to hide the existence of a physical console with real, physical games. So let’s try something a bit closer:The Pokémon anime. One of the episodes caused severe seizures to a large number of people who viewed it. Nintendo’s response? Stop running the episode, and never put Porygon in the anime again (because Porygon’s explosion is what triggered the seizures.) That was as far as they went:An archive of the entire episode still exists on the internet. Seems like Nintendo didn’t really want to hide the episode’s existence... (Note both of these events happened in the 90’s, around the same time as Super Mario 64.)

So, if this was just a case of the AI not working as intended, and rarely causing harm to those unlucky enough to get experimented on and hit with stroke-like symtpoms, Nintendo would likely respond similarly. Take the AI out of all future releases.. but make no effort to hide the existence of the AI. This is not what happened.

Secondly, some anomalies have been reported that both cause negative effects and cannot simply be explained as the AI attempting to learn something about you from it. (We’ll learn more about these anomalies later on. For now, the first reason is enough.)

That means that this is much more than just simple experimentation with procedural generation. If it wasn’t, Nintendo wouldn’t have responded the way they did.

But what more could it be?

In in order to find out, we’ll have to take a deeper look at the kinds of anomalies that can be found in the game. So we’ll be ending this intermission here, and returning to analyzing anomalies themselves. Eventually, we’ll come back to this topic.