|
|
 |
July 07, 2003
Self-promo: My Boston Globe essay on gender, language, and A.I.
As you may recall, a few months ago I wrote a short blog entry about the "he said, she said" software. A bunch of Israeli artificial-intelligence experts wrote a program that can automatically analyze a piece of anonymous text -- and tell you whether it was written by a man or a woman.
The Boston Globe's superb weekend "Ideas" section asked me to write a story about it, and it appeared in yesterday's section. It's online here, and a full copy is below!
He and she: What's the real difference?
According to a team of computer scientists, we give away our gender in our writing style
by Clive Thompson, 7/6/2003
Imagine, for a second, that no byline were attached to this article. Judging by the words alone, could you figure out if I were a man or a woman?
Moshe Koppel could. This summer, a group of computer scientists -- including Koppel, a professor at Israeli's Bar-Ilan University -- are publishing two papers in which they describe the successful results of a gender-detection experiment. The scholars have developed a computer algorithm that can examine an anonymous text and determine, with accuracy rates of better than 80 percent, whether the author is male or female. It's a computerized twist on an ancient debate: For centuries, linguists and cultural pundits have argued heatedly about whether men and women communicate differently. But Koppel's group is the first to create an actual prediction machine.
A rather controversial one, too. When the group submitted its first paper to the prestigious journal Proceedings of the National Academy of Sciences, the referees rejected it "on ideological grounds," Koppel maintains. "They said, 'Hey, what do you mean? You're trying to make some claim about men and women being different, and we don't know if that's true. That's just the kind of thing that people are saying in order to oppress women!' And I said 'Hey -- I'm just reporting the numbers."'
When they submitted their papers to other journals, the group made a significant tweak. One of the coauthors, Anat Shimoni, added her middle name "Rachel" to her byline, to make sure reviewers knew one member of the group was female. (The third scientist is a man, Shlomo Argamon.) The papers were accepted by the journals Literary and Linguistic Computing and Text, and are appearing over the next few months. Koppel says they haven't faced any further accusations of antifeminism.
The odd thing is that the language differences the researchers discovered would seem, at first blush, to be rather benign. They pertain not to complex, "important" words, but to the seemingly quotidian parts of speech: the ifs, ands, and buts.
For example, Koppel's group found that the single biggest difference is that women are far more likely than men to use personal pronouns -- "I", "you", "she", "myself", or "yourself" and the like. Men, in contrast, are more likely to use determiners -- "a," "the," "that," and "these" -- as well as cardinal numbers and quantifiers like "more" or "some." As one of the papers published by Koppel's group notes, men are also more likely to use "post-head noun modification with an of phrase" -- phrases like "garden of roses."
It seems surreal, even spooky, that such seemingly throwaway words would be so revealing of our identity. But text-analysis experts have long relied on these little parts of speech. When you or I write a text, we pay close attention to how we use the main topic-specific words-such as, in this article, the words "computer" and "program" and "gender." But we don't pay much attention to how we employ basic parts of speech, which means we're far more likely to use them in unconscious but revealing patterns. Years ago, Donald Foster, a professor of English at Vassar College, unmasked Joe Klein as the author of the anonymous book Primary Colors, partly by paying attention to words like "the" and "and," and to quirks in the use of punctuation. "They're like fingerprints," says Foster.
To divine these subtle patterns, Koppel's team crunched 604 texts taken from the British National Corpus, a collection of 4,124 documents assembled by academics to help study modern language use. Half of the chosen texts were written by men and half by women; they ranged from novels such as Julian Barnes's Talking It Over to works of nonfiction (including even some pop ephemera, such as an instant-biography of the singer Kylie Minogue). The scientists removed all the topic-specific words, leaving the non-topic-specific ones behind.
Then they fed the remaining text into an artificial-intelligence sorting algorithm and programmed it to look for elements that were relatively unique to the women's set and the men's set. "The more frequently a word got used in one set, the more weight it got. If the word 'you' got used in the female set very often and not in the male set, you give it a stronger female weighting," Koppel explains.
When the dust settled, the researchers wound up zeroing in on barely 50 features that had the most "weight," either male or female. Not a big group, but one with ferocious predictive power: When the scientists ran their test on new documents culled from the British National Corpus, they could predict the gender of the author with over 80-percent accuracy.
It may be unnerving to think that your gender is so obvious, and so dominates your behavior, that others can discover it by doing a simple word-count. But Koppel says the results actually make a sort of intuitive sense. As he points out, if women use personal pronouns more than men, it may be because of the old sociological saw: Women talk about people, men talk about things. Many scholars of gender and language have argued this for years.
"It's not too surprising," agrees Deborah Tannen, a linguist and author of best-sellers such as You Just Don't Understand: Women and Men in Conversation. "Because what are [personal] pronouns? They're talking about people. And we know that women write more about people." Also, she notes, women typically write in an "involved" style, trying to forge a more intimate connection with the reader, which leads to even heavier pronoun use. Meanwhile, if men are writing more frequently about things, that would explain why they're prone to using quantity words like "some" or "many." These differences are significant enough that even when Koppel's team analyzed scientific papers -- which would seem to be as content-neutral as you can get-they could still spot male and female authors. "It blew my mind," he says.
But this gender-spotting eventually runs into a $64,000 conceptual question: What the heck is gender, anyway? At a basic level, Koppel's group assumes that there are only two different states-you're either male or female. ("Computer scientists love a binary problem," as Koppel jokes.) But some theorists of gender, such as Berkeley's Judith Butler, have argued that this is a false duality. Gender isn't simply innate or biological, the argument goes; it's as much about how you act as what you are.
Tannen once had a group of students analyze articles from men's and women's magazines, trying to see if they could guess which articles had appeared in which class of publication. It wasn't hard. In men's magazines, the sentences were always shorter, and the sentences in women's magazines had more "feeling verbs," which would seem to bolster Koppel's findings. But here's the catch: The actual identity of the author didn't matter. When women wrote for men's magazines, they wrote in the "male" style. "It clearly was performance," Tannen notes. "It didn't matter whether the author was male or female. What mattered was whether the intended audience was male or female."
Critics charge that experiments in gender-prediction don't discover inalienable male/female differences; rather, they help to create and exaggerate such differences. "You find what you're looking for. And that leads to this sneaking suspicion that it's all hardwired, instead of cultural," argues Janet Bing, a linguist at Old Dominion University in Norfolk, Va. She adds: "This whole rush to categorization usually works against women." Bing further notes that gays, lesbians, or transgendered people don't fit neatly into simple social definitions of male or female gender. Would Koppel's algorithm work as well if it analyzed a collection of books written mainly by them?
Koppel enthusiastically agrees it's an interesting question -- but "we haven't run that experiment, so we don't know." In the end, he's hoping his group's data will keep critics at bay. "I'm just reporting the numbers," he adds, "but you can't be careful enough."
Clive Thompson writes for Wired, The New York Times Magazine, and The Washington Post.
Trackback Pings
TrackBack URL for this entry: http://www.collisiondetection.net/mt3/mt-tb.cgi/426
Listed below are links to weblogs that reference Self-promo: My Boston Globe essay on gender, language, and A.I.:
I'd like to see this same analysis done on non-English texts and see what comes up.
Posted by: Jeff Liu at July 7, 2003 4:31 PM
Yeah, that's a good point. A couple of people wrote in to the Boston Globe to ask that also!
Posted by: Clive at July 7, 2003 6:20 PM
Very interesting. The question "what is gender" is a bit of a canard. If people have been socialized as men or women, under the two categories that seem to organize our social life, they will write and speak differently. Should there ever come a time when gays or lesbians are allowed to express themselves at a very young age, pehaps as separate sociolect will emerge for them as well. But I don't imagine that such a thing exists now.
"Critics charge that experiments in gender-prediction don't discover inalienable male/female differences; rather, they help to create and exaggerate such differences. "You find what you're looking for. And that leads to this sneaking suspicion that it's all hardwired, instead of cultural," argues Janet Bing, a linguist at Old Dominion University in Norfolk, Va. "
This statement is the inverse of that which it attacks. There is nothing in the experiment you described that even remotely suggested that there is a genetic component to the linguistic phenomenon that the researchers found. They found a correlation, not a causation -- but the cultural constructivist dogma, kind of like a Pavlovian salivation effect, kicked in the minute someone did anything resembling scientific research into gender. I suppose that shelves full of slack cultural criticism , heavy on jargon but unbolstered by empirical evidence, is preferable to research.
Why did you waste valuable column space on this third rate commentary. You could have written it yourself and saved the phone bill -- YOU know how the rhetorical machine operates.
She adds: "This whole rush to categorization usually works against women." Bing further notes that gays, lesbians, or transgendered people don't fit neatly into simple social definitions of male or female gender. Would Koppel's algorithm work as well if it analyzed a collection of books written mainly by them?
Posted by: Erik at July 8, 2003 1:30 PM
ooopss ... hope you can edit this Clive, sorry.
My computer is acting wonky
Posted by: Erik at July 8, 2003 1:44 PM
Sorry -- took me a while to debug Movable Type to get the "delete" thing working. It's done now!
I dunno -- I don't think Bing's point is quite so intellectually shoddy. She's not saying that the science isn't correct, or that the scientists didn't find what they found. She's just saying that the framing device of the study is itself ideologically loaded, in gender-studies terms, because it starts off by trying to find differences between men and women's use of language. The point being that you could choose any two groups and also find differences -- such as, say, old and young people. Moreover, you could divide things up into a different granularity -- say, by looking for differences between how people write aged 15 to 25, 25 to 40, and 40 to 65. You could arguably do the same thing with gender, too, is Bing's point, in which case the decision to divide things up male/female is ideological. It's pretty common-sense idelogical, mind you, insofar as a lot of people would say, well, hell, yes, there really are only two major genders. But they're not unanimous about that, and Bing's a dissenter, and academic debate is pretty much all about debate so obtuse and hair-splitting that it wouldn't be tolerated anywhere else except a university. Which is what makes it fun, bien sur!
And interestingly, that age/gender comparison is interesting. Nobody would find it surprising if we could divine patterns in the way young people, middle-aged people, and old folks used language. After all, it's pretty cultural. Gender stuff is more loaded, because inevitably the public is sort of half-convinced that any cognitive differences between men and women are genetic. It's a nice wade into that grand old swamp of nature vs. nuture, heh. And that's why a) research of this sort invariable steps into political territory, and why b) I figured it'd be fun to see what critics have to say about it.
I hope it didn't look like I was attempting to debunk the A.I. research! I actually found it crazily interesting.
Posted by: Clive at July 9, 2003 7:36 PM
"But this gender-spotting eventually runs into a $64,000 conceptual question: What the heck is gender, anyway?"
Actually, there's no question at all. Sex is biological, gender is cultural. Sex is why teenage girls start having periods, gender is why having a period is such a big deal. Another example: sex differences are why men and women have different responses to certain medicines, gender differences are why men are less likely to follow a doctor’s orders and take that medicine.
It’s simple.
Keep confusing these terms and the next thing you know, you'll be using non-words like "impactfullness."
Posted by: The Tri-County ‘Maphrodite at July 10, 2003 12:25 PM
Good comment.
And you know, I'm pretty sure I've actually heard people *use* that word.
Posted by: Clive at July 14, 2003 1:32 PM
Posted by: Michael at November 29, 2003 6:44 AM
Posted by: american roulette at January 7, 2004 12:41 AM
Posted by: american roulette at January 7, 2004 12:41 AM
Posted by: Online Casino at January 16, 2004 2:49 AM
When Batman went home at the end of a night spent fighting crime, he put on a suit and tie and became Bruce Wayne. When Clark Kent saw a news story getting too hot, a phone booth hid his change into Superman. When you're programming, all the variables you juggle around are doing similar tricks as they present one face to you and a totally different one to the machine.
Posted by: Roger at January 19, 2004 6:47 PM
These secret identities serve a variety of purposes, and they help us to understand how variables work. In this lesson, we'll be writing a little less code than we've done in previous articles, but we'll be taking a detailed look at how variables live and work.
Posted by: Cassandra at January 19, 2004 6:47 PM
Let's see an example by converting our favoriteNumber variable from a stack variable to a heap variable. The first thing we'll do is find the project we've been working on and open it up in Project Builder. In the file, we'll start right at the top and work our way down. Under the line:
Posted by: Roman at January 19, 2004 6:47 PM
Inside each stack frame is a slew of useful information. It tells the computer what code is currently executing, where to go next, where to go in the case a return statement is found, and a whole lot of other things that are incredible useful to the computer, but not very useful to you most of the time. One of the things that is useful to you is the part of the frame that keeps track of all the variables you're using. So the first place for a variable to live is on the Stack. This is a very nice place to live, in that all the creation and destruction of space is handled for you as Stack Frames are created and destroyed. You seldom have to worry about making space for the variables on the stack. The only problem is that the variables here only live as long as the stack frame does, which is to say the length of the function those variables are declared in. This is often a fine situation, but when you need to store information for longer than a single function, you are instantly out of luck.
Posted by: Machutus at January 19, 2004 6:47 PM
These secret identities serve a variety of purposes, and they help us to understand how variables work. In this lesson, we'll be writing a little less code than we've done in previous articles, but we'll be taking a detailed look at how variables live and work.
Posted by: Felix at January 19, 2004 6:47 PM
Let's take a moment to reexamine that. What we've done here is create two variables. The first variable is in the Heap, and we're storing data in it. That's the obvious one. But the second variable is a pointer to the first one, and it exists on the Stack. This variable is the one that's really called favoriteNumber, and it's the one we're working with. It is important to remember that there are now two parts to our simple variable, one of which exists in each world. This kind of division is common is C, but omnipresent in Cocoa. When you start making objects, Cocoa makes them all in the Heap because the Stack isn't big enough to hold them. In Cocoa, you deal with objects through pointers everywhere and are actually forbidden from dealing with them directly.
Posted by: Watkin at January 19, 2004 6:47 PM
Inside each stack frame is a slew of useful information. It tells the computer what code is currently executing, where to go next, where to go in the case a return statement is found, and a whole lot of other things that are incredible useful to the computer, but not very useful to you most of the time. One of the things that is useful to you is the part of the frame that keeps track of all the variables you're using. So the first place for a variable to live is on the Stack. This is a very nice place to live, in that all the creation and destruction of space is handled for you as Stack Frames are created and destroyed. You seldom have to worry about making space for the variables on the stack. The only problem is that the variables here only live as long as the stack frame does, which is to say the length of the function those variables are declared in. This is often a fine situation, but when you need to store information for longer than a single function, you are instantly out of luck.
Posted by: Fulk at January 19, 2004 6:47 PM
For this program, it was a bit of overkill. It's a lot of overkill, actually. There's usually no need to store integers in the Heap, unless you're making a whole lot of them. But even in this simpler form, it gives us a little bit more flexibility than we had before, in that we can create and destroy variables as we need, without having to worry about the Stack. It also demonstrates a new variable type, the pointer, which you will use extensively throughout your programming. And it is a pattern that is ubiquitous in Cocoa, so it is a pattern you will need to understand, even though Cocoa makes it much more transparent than it is here.
Posted by: Silvester at January 19, 2004 6:48 PM
Let's see an example by converting our favoriteNumber variable from a stack variable to a heap variable. The first thing we'll do is find the project we've been working on and open it up in Project Builder. In the file, we'll start right at the top and work our way down. Under the line:
Posted by: Pierce at January 19, 2004 6:48 PM
Seth Roby graduated in May of 2003 with a double major in English and Computer Science, the Macintosh part of a three-person Macintosh, Linux, and Windows graduating triumvirate.
Posted by: Ciriacus at January 19, 2004 6:48 PM
Posted by: julia at January 24, 2004 8:06 PM
8914 check out the hot blackjack at http://www.blackjack-p.com here you can play blackjack online all you want! So everyone ~SMURKLE~
Posted by: blackjack at August 22, 2004 11:01 PM
???????????????????????????????????????????????????/>
????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? Search Google http://www.google.com/
Posted by: Google at November 11, 2004 7:29 AM
Posted by: FETISH FARMANIMAL SEX OF INCEST BEASTIALITYSEX at November 14, 2004 7:56 AM
Posted by: SEX BEASTIALITY BEASTIALITY PICS ONLY at November 19, 2004 12:18 PM
Post a comment
| | |
I'd like to see this same analysis done on non-English texts and see what comes up.
Posted by: Jeff Liu at July 7, 2003 4:31 PM
Yeah, that's a good point. A couple of people wrote in to the Boston Globe to ask that also!
Posted by: Clive at July 7, 2003 6:20 PM
Very interesting. The question "what is gender" is a bit of a canard. If people have been socialized as men or women, under the two categories that seem to organize our social life, they will write and speak differently. Should there ever come a time when gays or lesbians are allowed to express themselves at a very young age, pehaps as separate sociolect will emerge for them as well. But I don't imagine that such a thing exists now.
"Critics charge that experiments in gender-prediction don't discover inalienable male/female differences; rather, they help to create and exaggerate such differences. "You find what you're looking for. And that leads to this sneaking suspicion that it's all hardwired, instead of cultural," argues Janet Bing, a linguist at Old Dominion University in Norfolk, Va. "
This statement is the inverse of that which it attacks. There is nothing in the experiment you described that even remotely suggested that there is a genetic component to the linguistic phenomenon that the researchers found. They found a correlation, not a causation -- but the cultural constructivist dogma, kind of like a Pavlovian salivation effect, kicked in the minute someone did anything resembling scientific research into gender. I suppose that shelves full of slack cultural criticism , heavy on jargon but unbolstered by empirical evidence, is preferable to research.
Why did you waste valuable column space on this third rate commentary. You could have written it yourself and saved the phone bill -- YOU know how the rhetorical machine operates.
She adds: "This whole rush to categorization usually works against women." Bing further notes that gays, lesbians, or transgendered people don't fit neatly into simple social definitions of male or female gender. Would Koppel's algorithm work as well if it analyzed a collection of books written mainly by them?
Posted by: Erik at July 8, 2003 1:30 PM
ooopss ... hope you can edit this Clive, sorry.
My computer is acting wonky
Posted by: Erik at July 8, 2003 1:44 PM
Sorry -- took me a while to debug Movable Type to get the "delete" thing working. It's done now!
I dunno -- I don't think Bing's point is quite so intellectually shoddy. She's not saying that the science isn't correct, or that the scientists didn't find what they found. She's just saying that the framing device of the study is itself ideologically loaded, in gender-studies terms, because it starts off by trying to find differences between men and women's use of language. The point being that you could choose any two groups and also find differences -- such as, say, old and young people. Moreover, you could divide things up into a different granularity -- say, by looking for differences between how people write aged 15 to 25, 25 to 40, and 40 to 65. You could arguably do the same thing with gender, too, is Bing's point, in which case the decision to divide things up male/female is ideological. It's pretty common-sense idelogical, mind you, insofar as a lot of people would say, well, hell, yes, there really are only two major genders. But they're not unanimous about that, and Bing's a dissenter, and academic debate is pretty much all about debate so obtuse and hair-splitting that it wouldn't be tolerated anywhere else except a university. Which is what makes it fun, bien sur!
And interestingly, that age/gender comparison is interesting. Nobody would find it surprising if we could divine patterns in the way young people, middle-aged people, and old folks used language. After all, it's pretty cultural. Gender stuff is more loaded, because inevitably the public is sort of half-convinced that any cognitive differences between men and women are genetic. It's a nice wade into that grand old swamp of nature vs. nuture, heh. And that's why a) research of this sort invariable steps into political territory, and why b) I figured it'd be fun to see what critics have to say about it.
I hope it didn't look like I was attempting to debunk the A.I. research! I actually found it crazily interesting.
Posted by: Clive at July 9, 2003 7:36 PM
"But this gender-spotting eventually runs into a $64,000 conceptual question: What the heck is gender, anyway?"
Actually, there's no question at all. Sex is biological, gender is cultural. Sex is why teenage girls start having periods, gender is why having a period is such a big deal. Another example: sex differences are why men and women have different responses to certain medicines, gender differences are why men are less likely to follow a doctor’s orders and take that medicine.
It’s simple.
Keep confusing these terms and the next thing you know, you'll be using non-words like "impactfullness."
Posted by: The Tri-County ‘Maphrodite at July 10, 2003 12:25 PM
Good comment.
And you know, I'm pretty sure I've actually heard people *use* that word.
Posted by: Clive at July 14, 2003 1:32 PM
yes
Posted by: Michael at November 29, 2003 6:44 AM
Great work, keep it up!
Posted by: american roulette at January 7, 2004 12:41 AM
Great work, keep it up!
Posted by: american roulette at January 7, 2004 12:41 AM
Nice site. thx.
Posted by: Online Casino at January 16, 2004 2:49 AM
When Batman went home at the end of a night spent fighting crime, he put on a suit and tie and became Bruce Wayne. When Clark Kent saw a news story getting too hot, a phone booth hid his change into Superman. When you're programming, all the variables you juggle around are doing similar tricks as they present one face to you and a totally different one to the machine.
Posted by: Roger at January 19, 2004 6:47 PM
These secret identities serve a variety of purposes, and they help us to understand how variables work. In this lesson, we'll be writing a little less code than we've done in previous articles, but we'll be taking a detailed look at how variables live and work.
Posted by: Cassandra at January 19, 2004 6:47 PM
Let's see an example by converting our favoriteNumber variable from a stack variable to a heap variable. The first thing we'll do is find the project we've been working on and open it up in Project Builder. In the file, we'll start right at the top and work our way down. Under the line:
Posted by: Roman at January 19, 2004 6:47 PM
Inside each stack frame is a slew of useful information. It tells the computer what code is currently executing, where to go next, where to go in the case a return statement is found, and a whole lot of other things that are incredible useful to the computer, but not very useful to you most of the time. One of the things that is useful to you is the part of the frame that keeps track of all the variables you're using. So the first place for a variable to live is on the Stack. This is a very nice place to live, in that all the creation and destruction of space is handled for you as Stack Frames are created and destroyed. You seldom have to worry about making space for the variables on the stack. The only problem is that the variables here only live as long as the stack frame does, which is to say the length of the function those variables are declared in. This is often a fine situation, but when you need to store information for longer than a single function, you are instantly out of luck.
Posted by: Machutus at January 19, 2004 6:47 PM
These secret identities serve a variety of purposes, and they help us to understand how variables work. In this lesson, we'll be writing a little less code than we've done in previous articles, but we'll be taking a detailed look at how variables live and work.
Posted by: Felix at January 19, 2004 6:47 PM
Let's take a moment to reexamine that. What we've done here is create two variables. The first variable is in the Heap, and we're storing data in it. That's the obvious one. But the second variable is a pointer to the first one, and it exists on the Stack. This variable is the one that's really called favoriteNumber, and it's the one we're working with. It is important to remember that there are now two parts to our simple variable, one of which exists in each world. This kind of division is common is C, but omnipresent in Cocoa. When you start making objects, Cocoa makes them all in the Heap because the Stack isn't big enough to hold them. In Cocoa, you deal with objects through pointers everywhere and are actually forbidden from dealing with them directly.
Posted by: Watkin at January 19, 2004 6:47 PM
Inside each stack frame is a slew of useful information. It tells the computer what code is currently executing, where to go next, where to go in the case a return statement is found, and a whole lot of other things that are incredible useful to the computer, but not very useful to you most of the time. One of the things that is useful to you is the part of the frame that keeps track of all the variables you're using. So the first place for a variable to live is on the Stack. This is a very nice place to live, in that all the creation and destruction of space is handled for you as Stack Frames are created and destroyed. You seldom have to worry about making space for the variables on the stack. The only problem is that the variables here only live as long as the stack frame does, which is to say the length of the function those variables are declared in. This is often a fine situation, but when you need to store information for longer than a single function, you are instantly out of luck.
Posted by: Fulk at January 19, 2004 6:47 PM
For this program, it was a bit of overkill. It's a lot of overkill, actually. There's usually no need to store integers in the Heap, unless you're making a whole lot of them. But even in this simpler form, it gives us a little bit more flexibility than we had before, in that we can create and destroy variables as we need, without having to worry about the Stack. It also demonstrates a new variable type, the pointer, which you will use extensively throughout your programming. And it is a pattern that is ubiquitous in Cocoa, so it is a pattern you will need to understand, even though Cocoa makes it much more transparent than it is here.
Posted by: Silvester at January 19, 2004 6:48 PM
Let's see an example by converting our favoriteNumber variable from a stack variable to a heap variable. The first thing we'll do is find the project we've been working on and open it up in Project Builder. In the file, we'll start right at the top and work our way down. Under the line:
Posted by: Pierce at January 19, 2004 6:48 PM
Seth Roby graduated in May of 2003 with a double major in English and Computer Science, the Macintosh part of a three-person Macintosh, Linux, and Windows graduating triumvirate.
Posted by: Ciriacus at January 19, 2004 6:48 PM
Posted by: julia at January 24, 2004 8:06 PM
8914 check out the hot blackjack at http://www.blackjack-p.com here you can play blackjack online all you want! So everyone ~SMURKLE~
Posted by: blackjack at August 22, 2004 11:01 PM
???????????????????????????????????????????????????/>
????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? Search Google http://www.google.com/
Posted by: Google at November 11, 2004 7:29 AM
"Thre peaceniks are just ahead of the curve.
FETISH FARMANIMAL SEX OF INCEST BEASTIALITYSEX http://www.mayeur.net/
Posted by: FETISH FARMANIMAL SEX OF INCEST BEASTIALITYSEX at November 14, 2004 7:56 AM
Never mind that the intrinsics version is also quite unreadable.
SEX BEASTIALITY BEASTIALITY PICS ONLY http://www.sweetexxxtacy.com/
Posted by: SEX BEASTIALITY BEASTIALITY PICS ONLY at November 19, 2004 12:18 PM