And they did. Wood and two 18-year-old classmates — Allyson Sgro and Melissa Leventhal — teamed up to run what they called the first-ever, genuine Turing test. They corralled two male and four female students to serve as human chatters. For their robot, they picked ALICE, the chat bot pioneered by AI researcher Richard Wallace, which had the advantage of being award-winning, open-source, and, putatively, female. Then, to gather a large sample of in-the-dark interrogators, the trio decided on a bit of misdirection: They put up a Web site saying that they were hosting a “gender-guessing game.”

For three hours one day this spring, they watched as participants from across the US logged onto the Web site to play. Each interrogator faced two rounds. In the first, they chatted simultaneously with a woman and with a man masquerading as a woman, in an effort to spot the real woman. In the second round, they encountered a woman and then ALICE, again trying to detect the actual female. Interrogators had five minutes to ask anything they wanted. (How long is your hair? Do you wear lipstick?) The Simon’s Rock researchers did not warn interrogators that a bot was participating — they only hinted that there was a “secret element.”

As streams of chat flowed down the screens at Simon’s Rock, Wood, Sgro, and Leventhal looked carefully for signs of “bot awareness” — any comment by interrogators showing they suspected they were talking to a robot. Indeed, several people were onto ALICE, particularly after she — it? — coughed up weird answers:

Research subject: “Are you a computer?”

Computer: “Would it matter to you if I were metal instead of flesh?”

Yet in terms of Turing success, ALICE had one of her best days ever. Of the 42 people who completed both rounds, 23 never suspected she wasn’t a real woman — or if they did, they didn’t reveal it. Why did the bot do so well? Because, as Wood’s team realized, the Turing test includes a brilliant social hack. By forcing the male chatters to pretend to be something they’re not, the men performed awkwardly and gracelessly — kind of like a bot. Both ALICE and the men got tripped up by the question “What size panty hose do you wear?” None knew the sizes are A, B, C, and Q, not S, M, and L.

As a gay man who spent nearly his whole life in the closet, Turing must have been keenly aware of the social difficulty of constantly faking your real identity. And there’s a delicious irony in the fact that decades of AI scientists have chosen to ignore Turing’s gender-twisting test — only to have it seized upon by three college-age women, hyper-literate in the world of IM and discussion boards, where everyone is always sussing out your gender by studying the way you type.

“I used to have a handle that was based on the god Loki,” Sgro says, “so most people assumed I was a man for a very long time. Some of them never knew I was a woman until I met them, and they were like, Oooh.” Wood chimes in with her own story: “I used to work in engineering before I went back to school, and everyone always sent me email saying, ‘Dear Mr. Wood,’ because they didn’t think there were any female engineers.”

Yet their experiment may have left its own trail of deception. Participants were told to visit the guessing-game Web site the following day, when ALICE’s role would be revealed. But the trio of investigators suspects some people ALICE fooled will simply forget to check in. “They might never realize they were talking to a bot,” Wood says. “For the rest of their lives, they’ll have the memory of this day they talked to a wonderful girl.”