I was just trying yet again to explain why it's incoherent to think that robots might need rights or might become our progeny (beyond the extent to which they are – like novels – means for carrying our ideas to future generations). Suddenly I realised that AI is exactly like dreaming, but that may not be a useful analogy if you don't really understand dreams.
How dreaming works
|This wasn't actually a dream, it was a cactus in|
a carpark in Sedona, Arizona.
Dreams are your brain running stimulation across bits of your memory. There are lots of theories about why it might do this, like it may help you explore new ways to handle problems you've seen, or help you consolidate episodic memories into useful general principles. But for the moment all I'm trying to draw attention to is that dreams can briefly look enough like real life to fool you into thinking you're awake, yet they can also turn left into total craziness. Why? They look like real life because they are built out of memories of real life – they use the neural structures you've built up concerning how to understand real life. But they are incoherent because they are not being driven by actual perceptions of the world.
How AI is the same
When we use machine learning to program some part of AI, we are basically reusing computation humans have already done. So of course it is about our experience and looks like our experience. That's why you can type a few words into a search engine, and generally get the document you want. It's not because AI shares our experience of the world; it's because we build giant indices into records of our experience, and whatever you look up with such an index, of course you get back some record of human experience. But that doesn't mean that an AI system has anything like a human experience itself, it just contains records of our experiences. (Even our negative stereotypes.)
Thinking AI needs rights (in itself) is like thinking dreams are always true. AI may contain records we need to protect, and dreams may contain facts you observed before but hadn't really noticed yet. But that doesn't mean dreams are a supernatural means to generate new truths, nor that AI is a natural being having human-like experiences.