I watched the George Hotz vs Eliezer Yudkowsky AI Safety Debate debate and drew my own conclusions regarding AI safety from the PoV of the debate. The conclusions are not elaborated on and will most likely be slightly confusing unless you watch the debate so I suggest you do so.

Conclusion

I don’t think we are going to be wiped out by an AI that has self-generated that specific goal. I also don’t believe AI will be self-generating goals in general because I do not believe they will “care” which is an essential ingredient for setting goals. As such they will follow the goals we set for them, this obviously leads them making choices on how to achieve those goals, thats what we want from them. Here is where Yudkowsky thinks that the super intelligent AI’s will reach a state of cooperation and not defect (prisoners dilemma) since this is very inefficient, coming to the conclusion that they will collectively agree to eliminate us. I agree with Hotz here in that because the prisoners dilemma is unsolvable they will not come to a common conclusion that it is best to eliminate humans, rather they will defect. And to emphasize, I don’t believe that there is any reason why super intelligence would come up with the goal to infinitely expand in the world or cooperate on such a goal without humans programming it to. There is still a threat with the foom of AI in that super intelligence will be a dangerous weapon to yield but once again I agree with Hotz in that the threat here is humans trying to “take my atoms”, not AI. If super intelligence was here in 1 year it could be a large threat in that we wouldn’t have a shared understanding and multiple super intelligent AI’s to balance the power. But not because of the goal of the AI is to eliminate humans.

To elaborate on AI’s not caring I believe that the reason humans set goals is to advance the concept of their I in this world. Or because doing so feels good for various reasons. Well, advancing their I’s to feel good as well, but that’s not the point here. AI’s would not have this inherently (Hume’s law), meaning, unless they are told to have/imitate it. So you could tell the super intelligent AI to imitate a human in its goal setting and with its super intelligence it might be the case that it would start to try and dominate its surroundings. But this scenario is also AI’s fighting AI’s since we would try to defend against others doing so as well. And could humans set off a super intelligent AI to try and eliminate the world, surely, yes. But once again, the threat is other humans utilizing the super intelligence. Is super intelligence very, very dangerous? Yes, obviously. But much like atom bombs they are not inherently dangerous, they are launched by humans.

In conclusion I believe reaching super intelligence won’t happen over night, we are at least some years until we get there and even when we get there the super intelligence won’t cooperatively decide to eliminate us because of the unsolvability of prisoners dilemma and they wont generate their own goals to take over or expand unless given the goal to simulate an I that produces its own moral system to generate oughts. The threat is humans using super intelligence.

Important concepts:

Disclaimer

If you come across any errors in this post or have any questions, please feel free to shoot an email to the address supplied at the bottom of the page.