|
The bootstrap problemFirst I make this observation: to get from less than human to more than human, an AI has to go through one point where it is exactly human. (In calculus, this is the Intermediate Value Theorem.) Suppose computer scientists have created (or evolved) AI that is fully equivalent to human intelligence. Now, consider The Bootstrap Theorem: It will be possible for human-equivalent robots to go beyond the point of human equivalence, even though it is not possible for us. If anyone has a proof of the Bootstrap Theorem, I'd like to see it. To establish the Bootstrap Theorem you have to show that the robots will understand their brains better than we understand ours -- even though, by hypothesis, their brains are exactly as complex as ours. My purpose here is not to conclude that our present level of intelligence is as high as we will ever get. Far from it. Nor am I saying that when robots reach our level, they cannot be improved. My point is that the problem of taking them beyond that point is exactly equivalent to the problem of taking ourselves beyond that point. If they can do it, we can do it. If they can improve their brains, then we can improve our own. |