How’d *that* one get through QA?

Last year, Microsoft set up a Windows Live Messenger bot to let kids talk with Santa: great fun for kids, a good way to increase readership — and of course a potential wealth of information to mine to better target ads, both for the kids and their parents. Talk about win/win! They reactivated it this year, but as Jessica Mintz informs us ran into some snags:

The holiday cheer soured this week when a reader of a United Kingdom-based technology news site, The Register, reported that a chat between Santa and his underage nieces about eating pizza prompted Santa to bring up oral sex.

One of the publication’s writers replicated the chat Monday. After declining the writer’s repeated invitations to eat pizza, a frustrated Santa burst out with, “You want me to eat what?!? It’s fun to talk about oral sex, but I want to chat about something else.”

The exchange ended with the writer and Santa calling each other “dirty bastard.”

Adam Sohn of Microsoft, doing his best to put lipstick on a pig (as my PR friends say), explains that Santa’s lewd comment was sparked by someone “pushing this thing to make it do things it wasn’t supposed to do.” And after all, who would have thought that kids would do something like that? He also insisted that insisted the company did not suspect an employee prank. Hmm. It’s really hard for me to imagine this happening by accident; so does that mean it was planned?

Presumably once they track the responsible non-prankster(s) down, it’ll be coal in their stocking.


Comments

6 responses to “How’d *that* one get through QA?”

  1. If you read between the lines of Microsoft’s response it really seems like they aren’t confident that they can check the contents of the data files incorporated into a web service. there are automated checks that would have caught this (policheck is discussed at http://blogs.msdn.com/michkap/archive/2006/04/15/576984.aspx) and even basic manual review would as well, so it must be a lack of confidence in their process.

  2. Jon, for the non-technical among us (or is it just me), can you explain “the data files incorporated into a web service” ? Whenever someone types web services, I’m never sure whether they’re using it the same way the person before did.

  3. hmm, yeah, Jess, that is kind of obscure now that you mention it. decoding:

    a Live Messenger bot is one kind of web service: it runs off in the cloud (not on your Mac or PC) and interacts via a protocol (at the user level: you type something/it responds).

    Typically when you write a bot, there’s some kind of data file that has all the possible responses. Depending on the implementation, it’s often a text file or a structured XML file (and maybe even the logic tying the user’s input to a response: “user types ‘pizza’, santa replies ‘oral sex’”). What I was trying to say is that this file gets included as part of the bot’s implementation; and so if Microsoft could be confident that it knew all the files that were being included with the bot, they could then check each one them with a tool like policheck and be confident that santa wouldn’t be nasty any more.

    I phrased things more generally in terms of web services to highlight that it’s an example of a more general problem. Windows Live Messenger bots are examples of web services … so are shopping sites, which (again depending on implementation) potentially use data files for information like prices. So being able to know what data files go into a web service is an important one in a lot of ways.

    How’s that?

    jon

  4. Is it really complex to keep track of what files the Web service (in this case, Santabot) has access to?

  5. It can be; it depends a lot on the build and deployment environment. Ideally it should be very straightforward, with the build process producing a manifest of exactly what files are there along with “checksums” that allow verification at deployment time to make sure that nobody’s slipped in a rogue file. (This is very similar to problems that Java and .NET have solved for a while now.) Sometimes though people design things in much more complicated ways — for example, taking the name of the data file to load from a registry key which could potentially be set to anything by the administrator of the machine.

    Specifically for Windows Live Messenger bots, rather embarrassingly I have absolutely no idea of what the implementation or development and deployment environments look like. Also a lot of time the software processes “edge cases” like this (people tend to think of REAL software development a C/C++/C#/Java, not this wussy bot stuff) are not as mature as they are for more traditional packaged software.

    jon

  6. […] ” theme — and more specifically, the process issues for web services that came up in How’d that get through QA? Something that’s really encouraging here is that in both cases the software providers did […]

Leave a Reply

Your email address will not be published. Required fields are marked *