Such research should be outlawed. We (humans) obviously suffer from some kind of a suicidal instinct, when we're trying to build autonomous, self-aware and what's worse, even self-replicating machines.
All we need is one Skynet, then we're gonna be replaced by a more advanced intelligence.
I don't think it's possible or desirable to outlaw this.
As our understanding of neurobiology, neuroscience, consciousness, philosophy, computers, and mathematics increases, it will become easier and easier to simulate processes which are even more efficient (though probably much less robust) than the human brain at learning. I thing this is a good thing, because there are only two scenarios where there can be a serious problem to our dominance:
a) If a self-replicating robot is powerful enough to build itself up to the point where it can defeat all the armies of the world combined - which I don't think is possible (weapons research is not open to the public, and therefore not open to this hypothetical robot)
b) A program which spreads virally on the internet and uses the whole thing as a gigantic brain. Even in this case, simply shutting down the root servers will destroy its capacity. And for a program to actually do something like this, it must be able to mutate. If that happens, before it can reach any size large enough to control anything, it will have branched into mutually competing variants, thus posing no central threat.
And there is a trivially simple way of "keeping the drones in line" - program them to
enjoy serving their human masters, if we assume for a minute that we are going to build an underclass of robot slaves.
If we make Asimov's Three Laws of Robotics the foundation, we should have nothing to worry about:
1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
2. A robot must obey orders given to it by human beings except where such orders would conflict with the First Law.
3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
I want to find sarcasm in this, but keep getting reminded that there are people who truly beleive it.
