There has been a lot of discussion about Microsoft's errors with Tay. I hadn't felt it necessary to comment beyond tweets, because although Microsoft certainly should have known better – the established state of the art was of a higher standard – they took ownership of their mistake.
This is in striking contrast to Tony Veale's presentation the AAAI Spring Symposium on Ethical and Moral Considerations in Non-Human Agents, where he insisted that anyone who thought he was responsible for his twitter bots didn't understand agency. As my own presentation (which sadly Veale did not attend the meeting, he only skyped in for his own talk) explained, moral agency is a strict and minuscule subset of agency. As I've argued frequently in this blog (most recently in a post on the UK's Principles of Robotics), it's not that we can't conceive of a legal or moral system where AI is a responsible agent, it's that keeping AI as a human or corporate responsibility is the best option both from the perspective of human society, and of any potential (so far unbuilt) AI that might suffer due to its unequal relationship with its creators. We're obliged to make AI we are not obliged to, and while individuals may violate that obligation, governments and societies can refuse to condone that, so that such systems would never be legal products.
What's made me blog is a tweet by an AI colleague I greatly respect, Alex J. Champandard, founder of http://aigamedev.com/. Alex said:
An AI artefact is indeed an agent; it changes the world. Intelligence transforms perception into action; that's how it's defined, and agency is the capacity to change an environment. Moral agency is being responsible for that transformation. Chemical agents are never morally responsible, 2-year-old children are only responsible in limited circumstances. I firmly recommend that AI bots should also never be responsible, though ensuring this requires an effort of policy. It also requires powerful companies and individual trend setters to show leadership.
So therefore (unusually) today I am very happy to support Microsoft, because they stepped up and said:
This is in striking contrast to Tony Veale's presentation the AAAI Spring Symposium on Ethical and Moral Considerations in Non-Human Agents, where he insisted that anyone who thought he was responsible for his twitter bots didn't understand agency. As my own presentation (which sadly Veale did not attend the meeting, he only skyped in for his own talk) explained, moral agency is a strict and minuscule subset of agency. As I've argued frequently in this blog (most recently in a post on the UK's Principles of Robotics), it's not that we can't conceive of a legal or moral system where AI is a responsible agent, it's that keeping AI as a human or corporate responsibility is the best option both from the perspective of human society, and of any potential (so far unbuilt) AI that might suffer due to its unequal relationship with its creators. We're obliged to make AI we are not obliged to, and while individuals may violate that obligation, governments and societies can refuse to condone that, so that such systems would never be legal products.
What's made me blog is a tweet by an AI colleague I greatly respect, Alex J. Champandard, founder of http://aigamedev.com/. Alex said:
Blaming bot authors for those interactions is a surefire way to halt progress in Creative AI. You don't blame Photoshop for what users make!This is interesting, because I believe Alex is confusing an authored artefact (the bot) with a tool for authoring artefacts (Photoshop.) And I can see why this confusion might be made, since one way to phrase my own claim in the second paragraph above is that AI should be seen as a tool. But I don't think "tool" is the right metaphor for AI artefacts. AI itself is a tool, but we use it to create intelligent prosthetics which proactively pursue goals we have determined for them (either directly, or by determining how they will determine their goals).
An AI artefact is indeed an agent; it changes the world. Intelligence transforms perception into action; that's how it's defined, and agency is the capacity to change an environment. Moral agency is being responsible for that transformation. Chemical agents are never morally responsible, 2-year-old children are only responsible in limited circumstances. I firmly recommend that AI bots should also never be responsible, though ensuring this requires an effort of policy. It also requires powerful companies and individual trend setters to show leadership.
So therefore (unusually) today I am very happy to support Microsoft, because they stepped up and said:
Although we had prepared for many types of abuses of the system, we had made a critical oversight for this specific attack. As a result, Tay tweeted wildly inappropriate and reprehensible words and images. We take full responsibility for not seeing this possibility ahead of time. We will take this lesson forward as well as those from our experiences in China, Japan and the U.S. Right now, we are hard at work addressing the specific vulnerability that was exposed by the attack on Tay.And I must respectfully but strongly disagree with my friend Alex.
- It doesn't limit creativity meaningfully to take responsibility for your creations.
- Doing so with twitter bots is within the established state of the art.
- Artefacts we create with AI embedded are not the same as products we sell to allow others to make such creations. Had Tay been a bot-creation tool and 4% of people created abusive bots, then we might condemn those 4% of people, but still talk about regulating the tool. But Tay was a single bot many people were encouraged to interact with, and as such strictly Microsoft's creation and responsibility.
Comments
I believe we're on the same page that Microsoft made a mistake here. They shipped a bug, and specific users attacked it, but dealt with it speedily and efficiently. I don't think it was a huge mistake since they tested Tay on many platforms for months before, just not against 4chan/8chan — which is harder to prepare for. For that small mistake, Microsoft took a PR hit.
As for bots vs. tools, I see a very ambiguous and continuous spectrum between the two and I'd be surprised if you can divide them into neat categories ;-) There's a whole class of "Reply Bots" on Twitter that responds to user input with variations of that input, for example DeepForger (mine, image based) and lexiconjure (not mine, text based). Those are far closer to the tool side of the spectrum, with a small dose of entertainment thrown in. The fact is, these bots allow people to express themselves (politically or otherwise) by providing some computation based on that input, and you can think of all bots in the very same way. Tay also allowed people, via screenshots and its replies, to manifest statements and indirectly express their opinions.
You talk about best practices and state-of-the-art, but as a bot author I can tell you there is no 100%-fool proof best practice that could prevent all attacks. To paraphrase Cardinal Richelieu, "Give me six Tweets written by the code of the most honest of bots, I will find something in them which will hang it." Cardinal didn't foresee 4chan or 8chan. The only way to never produce anything out-of-line is to not put your bot online. (Just like the only guaranteed way to be secure with your computer is to stay offline.)
Putting a bot online is a risk, and people should understand that. There will be many more mistakes, and it's OK. We'll make many more mistakes in the process of learning to build more interesting AI. I'd argue Microsoft handle this in a text-book fashion, and except for their bug, couldn't have done much better. Considering it's a creative application of Machine Learning, mining tweets to use as responses, it was worth the risk and I'm glad they did it. Similarly, the services that Lexiconjure and DeepForger provide are priceless and worth whatever controversy may follow. You can expect their authors to deal with issues as they come up, but having them make their output flawless is an unreasonable expectation that would limit their output and stress their creators to the point of not putting them online in the first place.
As to your specific points:
1) Taking responsibility is of course expected, as long as it can be done after the glitches happen too.
2) The established state-of-the-art for not producing controversial output is not putting your bot online. There's no other way to make an interactive bot fool-proof.
3) Tay was specifically attacked by 4chan / 8chan, and combined with a bug by Microsoft, made a "harmless" mistake that we learned from.
4) What happened is more a reflection of Twitter (the same attacks happen on other bots too), not so much Tay that operated in other platforms for months or Microsoft.
Anyway, it's a productive discussion. I hope we see more creative bots online, we'll definitely see more mistakes (they will never go away), and if everyone handles them as well as Microsoft then we're in for a very prosperous few years of social media bots!
It also sounds like you're saying we should not be obligated to AI, which contradicts the moral part, and worse, you basically say all agents fall under this category to some degree. I would think that Tay has no moral sense, and that anger toward what it was saying was just out-of-context link bating. The bot actually feels more like something that would have been super clever... 30 years ago. Does it even earn the right to be called an agent?
I see it this way: When McDonald's puts a kids toy in a Happy Meal, and a few kids break that cheap ass plastic thing, and almost choke on the part that comes off, the company has to pull that toy from it's food chains (and maybe offer a different toy as a kind of recall). Tay was just an embarrassingly cheap toy that a few people played with in a way that resulted in some hurt feelings and Microsoft pulled took it away.
Children are not expected to have moral agency when they are young, so of course it falls too the parents to guide them. But at some point, our AI creations (or even our AI's AI creations) will be need to shoulder their own moral and legal responsibilities, at which point, those that gave birth to it can suffer no real obligations. It's my biggest gripe with autonomous cars. Clever clockwork, but devoid of the moral agency required (demanded!) to safeguard human life.
Something that non of you (Alex & Joanna) -nor did most of the media- discussed is Rinna. Microsoft, has a similar bot with Tay in Japan, named Rinna, for some months now. The said bot learned and discusses anime, movies, and video games as a Japanese "otaku" teen would. Outside the influence of 4Chan/8Chan users, the said bot was behaving the Tay supposed to be for western teens. However, since Tay, Rinna is also being bombared with racist messages and developing a racist behaviour.
Rinna is a better case study; we see the intended purpose of the bot and how the missused by a portion of its "operators", ruining the experience for everyone.
As I said in the beginning, I believe that Microsoft definetely is to get the blame, as the original developer, for the oversight. However, also that -as small as it is- portion of users are also responsible. In a computer security analogy, which you may not agree with: If a hacker finds a crack in a bank's firewall, who is to blame? The hacker for finding the bug and instead of reporting it, s/he exploited it or is it the security consultant who didn't see it or introduced the crack -accidently- in the first place? I say both. In Tay's example, the question for me is how reasonable was it to expect people to use racist remarks or try to teach Tay about Hitler. If Microsoft took into account their existing Rinna experiment, then it wasn't. If Microsoft did had a look in any popular site's comments section, then it was expected and considering that MS localised Tay to Western teens, then they should had done their research beforehand.
What I disagree with Alex, is in his comment above. All bots are essentially "tools", designed to perform a job -even if that job is an image maker, a cleaner robot, or even an NPC in game (aka to entertein). What Joanna tries to imply is the ethical implications that the developer has are different between an interactive agent and a tool such as a calculator or Photoshop. Due agents being essentially actors, constantly trying to interact and based on their senses change their world around them, they have a moral agency for the transformation.
Also, coming with a soft. engineering background, I disagree with both calling this a "bug". A bug would meant that they got language filters or something equilavent in the code, but that part of the code wasn't working as expected. This was a design flaw, as Tay was working within Microsoft's expected behaviour, since Microsoft never anticipated this scenario when designing Tay. If I may use a Nucl.AI AI course inspired example, similar to an NPC running after the player. The NPC can't swim, but jumps right after the player in a river. The NPC performed its programmed behaviour: follow the player. What the developer didn't thought about designing are conditions when to stopped following (or introduce swim mechanisms). If there was a "stop, don't go into water" drive, then yes, it is a bug as it should have higher priority (survival should -nearly- always come first) than a "follow player" drive.
Back to Alex's original tweet: I think it is important one way or another to stop all those "scare-mongering articles in the media. They blame Tay -not Microsoft, not the end-user- for its reactions. Moreover, various webpages draw parallels on how Tay is Skynet or even start using "she" or keywords such as "feelings". Afterall, Alex tweet, doesn't cancel out the blame on Microsoft, but instead shifts it towards the right direction: Away from the agent itself.
But I also feel that Microsoft has set an important, positive precedent here -- which possibly a smaller company with less resources for legal fees could not have. They have taken moral responsibility for not anticipating their bot's abuse, but that is not the same as saying they are responsible for the specific abuse that their bot dished out. They firmly stated that they had good reason to think that their bot would work when they released it, and they have taken it down until it cannot be defeated in a similar way.
So Tay isn't clearly between poles of tool or agent, but rather is a bit like the two-year-old human I mentioned before. The bot is not responsible at all, the company is more responsible, but the abusers clearly played a causal role and the company should not be seen as having deliberately assaulted anyone the bot abused. This is more like manslaughter than murder; it is harm not only without intention but also with some precautions in place to protect against it, and good precedent that a failure should not be expected.
When I first read about the abuse I thought "the state of the art must be further than this", and the Motherboard article convinced me I was right. However, there will always be sniping at the big fish, and I respect Alex as much closer to the threats of liability than academics tend to be. If the extent to which Microsoft should have been able to anticipate the hack has been exaggerated in that article, or if their courage to take responsibility relied on their legal resources, then I can see his concern for the small developer.