FREE counter and Web statistics from sitetracker.com
December 20, 2006
How YouTube is saving the lost art of guitar wanking









A while ago, one of the hottest videos on YouTube was of a young guitarist, sitting on his bed, playing a speed-metal version of Johann Pachelbel's "Canon in D". It was totally mesmerizing; I watched it about five times in a row, as did several million other fans. (That's a screenshot above.) Chuck Klosterman, the culture critic for Esquire, also got interested in the clips, and he realized that it presaged an intriguing cultural shift: YouTube is bringing back the lost art of guitar wankery.

Why? Because 1980s-style shredding faces problematic paradox: "Very often," as Klosterman pointed out, "profoundly exceptional guitar playing is boring to listen to." YouTube, however, changes those stakes, because it offers us a new way to see the craft at hand. As Klosterman wrote:

It's difficult for nonmusicians to appreciate world-class guitar playing through solely sonic means, mostly because a) the difference between great guitar playing and serviceable guitar playing is often subtle, and b) every modern listener assumes production tricks can manufacture greatness. (As a result, radio audiences are automatically skeptical of what they hear.) Guitar brilliance usually comes across as ponderous. But that changes dramatically when one adds the element of video; somehow, watching changes the experience of hearing. There are certain things that sound good only when (and if) you can see them. And YouTube lets you see them.

Whenever you enter the highest, stupidest, Bucketheadiest stratosphere of electrified insanity, one thing becomes clear: Guitar-godding is an athletic pursuit ... and athleticism needs to be seen in order to be appreciated ... Early in his career, Eddie Van Halen turned his back to the audience whenever he played solos, supposedly because he was afraid rivals would steal his techniques. Had he insisted on doing this forever, very few people would have cared about his music. (We would probably assume "Eruption" was performed on a German synthesizer built from the spare parts off a fire engine.) People needed to see how his fingers worked. Only then could they understand that Eddie Van Halen was doing something they could not understand. His guitar was not a primitive machine that made it easier to meet girls and get free drinks; his guitar was a futuristic machine that was fucking hard to fucking operate. You can fake being cool, but you can't fake being good. That's the musical potentiality of YouTube: It allows us to see elements of musicianship that are difficult to hear (even though hearing is supposed to be the whole idea). It could make a handful of people recognize (and care about) virtuosity in a way that hasn't happened since the fall of King Crimson.

Testify, brother. I think Klosterman's right, and I for one could not be happier. This is partly because I really enjoy speed metal, and indeed as a teenager attempted to play the stuff myself. (I was fast, but not that fast.) And I confess I've been increasingly dissatisifed with the direction of modern pop, which has more and more privileged screechy and/or whiny vocalists who are utterly unable to play any instrument themselves, and thus, usually, unable to actually write music or songs themselves.

It reminds me of that study that I blogged about a while ago, which showed that of all celebrities, musicians were the least narcissistic because the mere act of acquiring and honing a quantifiable technical skill had the psychological effect of pulling your head out of your ass, at least a slightly. I think the same is true of the pop audience: The more they're schooled to respect actual musicians with actual skills, maybe the less they'll be impressed by talentless whiny "singers".

Posted by Clive Thompson at 04:57 PM
The five-year weather forecast: My latest feature for New York magazine









A while ago, New York magazine asked me to write a big feature on the future of New York's weather. This plunged me hipdeep in all sorts of interesting stuff -- from the many naturally-recurring climate cycles that govern our weather, to the predicted effects of global warming. The result was published a few weeks ago, and I'm just now getting around to blogging it!

The story -- accompanied by that excellent illustration above, created by James Porto -- is free at the magazine's web site, and a permanent copy is also archived below!

The Five-Year Forecast
Unseasonably warm, with freakish snowfalls and chance of cyclone. This winter will be weird, and the weather will keep getting weirder.
by Clive Thompson

The ash tree vanishes.

The next time you're in Central Park, go up to Harlem Meer on the north end, then wander westward on the pathway into the heart of the park. After the first sharp turn, look off to the west and you'll see a thick stand of ash, its rough bark set off by delicate oval leaves. Long before New York existed, the ash thrived in this region, and the city's settlers used the tree's dense but springy wood to make everything from church pews to baseball bats. The ash has been here since the beginning.

But its time is about to come to an end. In recent years, foresters have quietly decided not to plant any new ash trees. Why? Because the city is becoming too warm and dry for them, and they're dying off. Green and white ash, our local varieties, are classified as "hardiness zone three or four," northern trees that prefer moist, well-drained soil. New York used to be like that, 200 years ago -- but the temperature in the past century has risen over two degrees, and it's getting drier every year. "Last year we had stretches without rain that were practically six weeks long," says Neil Calvanese, vice-president of operations for the Central Park Conservancy, which maintains the park. And the warmer weather has introduced new wood-eating bugs that afflict the tree. Normally an ash will live 250 years, but this summer Calvanese had to chop down a majestic 130-footer when it stopped thriving. "Ash in the park," he says, "I really don't see as having much of a future."


So he's decided to let the ash slowly die off. An urban forester has to think decades into the future, and the city's only going to get hotter and hotter. Instead of the ash, city foresters are starting to plant trees like the persimmon, which thrives in southern climates like Washington's or even Atlanta's. Because that's what the future of New York looks like, weather-wise: There will be fewer and fewer wooden baseball bats and church pews -- but plenty of reddish-purple persimmon fruit.

Nor is Calvanese alone in planning for the next 50 years. In September, while unveiling plans for a new city bureau, the Office of Long-term Planning and Sustainability, the mayor pledged to make climate-change issues a priority. It's not hard to understand why the idea finally occurred to Bloomberg. This summer, the weather was Topic A among New Yorkers, as we sweltered through a record-busting heat wave in July. You could see it in the worried faces of people as they marched miserably down the liquefying sidewalks this summer: Is this it? Has global warming finally arrived? Is this what we're in for -- permanently? The abrupt shift into a spookily mild fall, and especially the freakishly warm last few weeks, has only fanned the conversational fires.

If it seems like the city has been getting hotter for years, it's not your imagination: Six of the top-ten hottest summers in the city have occurred since 1990. (In fact, we haven't had a top-ten coolest summer since 1927.) But our summers aren't the only extreme weather, because our winters have been breaking records in precisely the other direction. The last four winters have all had snowfalls way above average, a string that we haven't seen since the 1800s. Worse, that record-breaking snow is coming in huge dumps -- not a flake in the sky for months and months, then a huge blizzard that sweeps through town, brings the city to a halt, and vanishes a week later.

The weather isn't just getting harsher: It's getting weirder. Here's why.


The little boy comes back.

If you had a satellite's view of the Earth this summer, you would have noticed something odd happening to the equatorial Pacific: The water was getting warmer.

In June and July, government satellites and thermal buoys that track the heat of the ocean detected a huge mass of bathtub-warm water forming off the coast of Indonesia and Sri Lanka. By August, it had broken loose and begun to migrate across the ocean, headed toward Peru. The normally chilly waters of the Pacific gradually warmed and warmed, until by September the water along the entire equator -- a mass twice the size of Europe -- was one long ribbon of warm ocean.

It could mean only one thing: El Nino had returned.

El Nimo -- Spanish for "the little boy" -- is one of the most powerful natural forces that affect global weather. An El Nino event begins when the trade winds in the Pacific break down. Normally they blow east to west, pushing the sun-warmed water on the surface of the Pacific toward Indonesia, where it piles up offshore and makes that nation particularly rainy and wet. Every two to seven years, though, for reasons that are still a bit mysterious, the trade winds falter. As they slacken, the trades' grip on this pool of warm ocean-surface water slackens, and it begins to spread across the entire Pacific. A really strong El Nino will warm the coast of Peru by as much as eight degrees.

Which is when the weird weather begins -- because all that moist, warm water evaporates into the air and produces spectacularly intense rain, winds, and storms. In the U.S., California is the first to get hit: The last big El Nino, in the winter of 1997–98, produced 40-foot-high waves, home- destroying landslides, and the 200-mph Hurricane Linda. The PGA amateur tournament was canceled when golf courses were reduced to mud pits; bewildered California fishermen began catching mahimahi and marlin, because the ocean became so warm that these normally tropical fish swam hundreds of miles north. "A really big El Nino," says Vern Kousky, a government meteorologist who first announced the El Nino that formed this summer, "has just a huge, huge effect on everything."

Yet the effect isn't all bad. Indeed, a strong El Nino makes New York a much nicer place to live -- balmier, and with less-violent weather. That's because as the hot, wet air flows in off the Pacific, it amplifies the speed and intensity of the jet stream, the rivers of fast-moving air that flow toward the east across the country at 20,000 feet. And this more-powerful jet stream, in turn, affects weather all the way to the East Coast. As the jet stream shoots out over the Atlantic, for example, it produces wind shear that "tears hurricanes apart before they can form," explains Phil Klotzbach, a hurricane predictor with the Tropical Meteorology Project at Colorado State University. Indeed, El Nino is probably the reason this year's hurricane season has been so much less punishing than last year's parade of tropical monsters.

What's more, the amplified jet stream scoops up the milder weather of Washington State and drags it across the country. The current El Nino isn't yet strong enough to have this effect, but Kousky predicts it could within a few months. "It takes Seattle's weather," Kousky says, "and brings it to New York." During the particularly intense El Nino of 1997–98, not a flake of snow remained on the ground in Manhattan that winter -- something that hadn't happened in more than 100 years. When you're sitting outside sipping a cappuccino on an unseasonably balmy March afternoon, you'll have El Nibo to thank.

You can also thank it for a lighter energy bill. Warm winters reduce the demand for oil, and the absence of killer storms keeps Gulf Coast oil-refinery production high. El Nino thus produces cheap oil -- indeed, ten years ago, it actually helped cut the price of a barrel in half. This year's El Nino won't have quite that powerful a kick, but it's already been credited with pushing the price of a barrel of oil below $60 for the first time in six months. This slump in prices means spectacularly bad news for the go-go hedge funds that have made their fortunes by betting on wartime oil shortages -- such as Amaranth Advisors, the $9 billion fund that lost a mind-bending $6.5 billion in two weeks, then shut down in September. As the weather swings, so swings the energy market.

The fifties are here again.

Digging deeper into the meteorological forces that undergird our weather, I arrange to meet Heidi Cullen -- the chief climatologist for the Weather Channel -- at a Tribeca cafe on a recent stunning fall day, and she told me about her father. Her family is from Staten Island, and that's where her father grew up in the fifties. The weather back then was a nonstop carnival of storms and hurricanes. The worst, Hurricane Donna in 1960, terrorized the entire Northeastern Seaboard, and by the time it wound up on Long Island, 364 were dead. So last year, when the tail ends of Katrina and a half-dozen other huge storms lashed New York, Cullen's father didn't find it particularly surprising.

"He was like, 'You guys have no idea what it was like. There were so many big storms and hurricanes back when I was young,'" Cullen says. Only the brave lived on waterfront property.

And, as it turns out, her father's weather has come back to haunt us: The storms of today eerily resemble those of the fifties. Why? Because of a climate pattern that is set to govern the skies of New York for the next twenty years: the Atlantic Multidecadal Oscillation (AMO).

The AMO is the enormously slow, powerful heartbeat of the Atlantic, a 70-year-long cycle in which the ocean slowly heats up and then cools down again. Scientists began to notice this pattern ten years ago, when they looked back over 150 years of records of the Atlantic's surface temperature. For a period of 35 years, it would warm up by about one to two degrees; then it would gradually decline until it had cooled by the same amount, over roughly the same time. That difference in temperature doesn't sound like much, but when you're talking about a body of water as massive as the Atlantic, a two-degree shift represents an enormous amount of energy. Indeed, hurricanes typically begin forming when ocean water hits 78 degrees Fahrenheit, so every extra degree counts.

The AMO has thus been quietly and secretly influencing New York life for decades -- most particularly our housing market. From the mid-sixties to the mid-nineties, the AMO was in its "cold" orientation. Hurricane activity quieted. That's why the seventies and eighties saw such a housing boom on the coastline of Long Island: The sea was colder but more forgiving, and as the brutal storms of the fifties receded, insurance rates for those houses decreased, making coastal life more affordable.

But in 1995, the AMO started to reverse polarity, and the Atlantic began warming up. Sure enough, our modern plague of hurricanes started right on cue. "People who live around here have a kind of short-term memory," says Cullen. "With the last few years of storms, they've been saying, 'What the hell is happening?' But this is exactly what you'd expect with a warmer AMO. We're back to where they were in the fifties."

And now the economics of the coastline are reversing themselves. More and more gorgeous waterfront mansions are incurring water damage from the increasingly intense storms. As insurance companies face bigger payouts, they're starting to pull out. This year, Allstate canceled 30,000 policies in coastal counties of New York, and it plans to limit new business in the region; Nationwide Mutual Insurance has stopped issuing new homeowner policies in parts of Long Island's storm-prone eastern end. One study suggested that the coastal exposure in the Greater New York area had doubled in the past ten years to $2 trillion in replacement value.

"That's trillion, with a t," marvels Michael R. Murray, assistant vice-president for financial analysis at the Insurance Services Office. "We're talking a massive amount of exposure. We're talking about something as big as Florida."


Hitting the funnel.

What's a region facing trillions in hurricane damage to do? Five years ago, Malcolm Bowman, an oceanography professor at suny Stony Brook, met with the heads of the Port Authority to propose an audacious project: a trio of massive hydraulic gates to protect New York from the huge storm surge in the event a catastrophic hurricane hit the city. Gargantuan gates would be anchored to the floors of three rivers: in the upper East River east of La Guardia, at the Verrazano- Narrows Bridge, and near Perth Amboy, New Jersey, off Staten Island. If a serious hurricane hit the city and ocean waters suddenly began rising, the barriers would close; their tops would stick 25 feet above the water's surface, preventing the Atlantic from flooding -- and destroying -- the city.

Building the barriers would be the most ambitious engineering project in the city's history. The Verrazano gates alone would have to stretch across a mile of open water and reach down 100 feet to the bottom. One design has them looking like saloon doors, swinging shut at the threat of danger. In another design, they lie flat on the bottom and swing upward. In either case, they would cost "at least $10 billion to erect," guesses Bowman.

Yet the unsettling fact is that we might need them pretty soon. Statistically speaking, Bowman points out, New York is overdue to be hit by an honest-to-goodness cyclone -- one that brews in the warm waters down South, then wanders up the Atlantic Coast and slams head-on into Manhattan. One of these monsters hits New York every sixteen to twenty years. We haven't had one since 1992, precisely fourteen years ago, which makes our continued luck more and more unlikely every year. What's more, a year from now, when El Nino ends, the AMO should reassert itself as a powerful driver of hurricanes -- meaning more wind and water damage is on the way.

New York is in an exquisitely bad position to survive a direct hit. The laws of physics are arrayed against us. The first problem is that while the colder ocean waters off our shores tend to dissipate hurricanes (which is good), the ones that do make it this far north are the ones that move the fastest (bad). When Katrina hit New Orleans, it was traveling at 15 mph. But the one that will eventually hit New York will be traveling at a speed closer to 60 mph. "That means that, effectively, a Category 2 hurricane hits you as hard as a Category 5 hurricane down South," says Nicholas Coch, a leading expert studying New York hurricanes at the School of Earth and Environmental Sciences of Queens College. "Everyone says, 'Oh, it's just a Category 2 storm!' But when you realize what it's going to do, your hair stands on end."

Worse, a hurricane that enters New York Harbor is boxed in. New Jersey and Long Island form an L shape that hurricane experts ominously call "the funnel." Unlike in Florida, where a hurricane pushes water up the coast, a hurricane that hits the funnel has no place to go. Its energy is concentrated into a tight blast that would hit Manhattan, driving the full force of the harbor's waters up the East and Hudson rivers. Winds hitting the city's skyscrapers would experience a "Bernoulli effect" amplifying the wind further, squirting a torrent of moving air into Central Park at tree-shredding intensity. "We're cursed by our geography," as Coch puts it. A hurricane that would be relatively mild if it hit anywhere else in the country would thus devastate New York.

In 1893, a Category 3 hurricane did in fact hit the funnel. Water levels rose thirteen feet in a single hour, then kept on climbing until they hit a maximum of 25 feet. If that happened today, transportation would grind to a halt, with both the FDR and the West Side Highway, much of the subway, and all traffic tunnels submerged. All airports would be shut, and, according to one study, upriver water-treatment systems would be contaminated by ocean saltwater.

Perhaps most remarkable, Manhattan would be split into two islands at Canal Street, which is low and would be completely flooded. "The financial district would be entirely cut off," as one group of Columbia University engineers concluded when it assessed the impact of a direct-hit hurricane. Even higher land uptown would not escape the damage: One computer model of a hurricane estimated that water would surge eighteen feet at West 96th Street.

By the time the storm subsided, the overall cost to New York would be at least $100 billion -- 100 times more than the cost of the average storm damage we get in a year now. Even Hurricane Floyd, which swept ashore in September 1999 and was considered a big deal at the time, caused a mere $1 billion in damage.

The only thing that could truly prevent this damage are those gates that Bowman has proposed. But marshaling the political will -- to say nothing of the billions of dollars -- to build them could be difficult. The initial meeting Bowman had with Port Authority officials back in 2001 seemed promising. "They agreed it was a big issue," he recalls.

The next week, however, was 9/11. Bowman shrugs. "And you know what happened next."


A $16 billion bet on a warmer winter.

"One third of our gross domestic product," declares Colt Heppe, an executive vice-president of ICAP, a new kind of trading desk, "is affected by the weather." The white-haired, bearlike broker is peering over the shoulder of Peter Rosen, one of his twentysomething traders, as Rosen frantically works a deal via instant messaging. Connecticut-based Tudor Investment Corporation is asking about buying a "weather option" -- a financial bet based on the ebb and flow of the weather. Is it going to get warmer? Colder? If your business suffers pain when the mercury unexpectedly plunges or soars, Heppe's there to offer intriguing financial instruments to hedge the risks.

"Predicting weather is still such a totally inexact science. Kind of like economics," Heppe jokes. "And that's why we exist."

The idea of weather futures has been around for decades, but it developed into an actual market only about ten years ago. The concept is fairly simple. Heppe divides the year into two five-month seasons -- a November-to-March winter and a May-to-September summer (April and October act as "shoulder" months bridging the seasons). He figures out the average temperature expected each day for fifteen American cities based on the year-round measurements taken at their airports, and by studying the daily weather predictions issued by U.S. government and European meteorologists.

Then he sells options or puts based on how much people think the actual temperature will deviate from that mean. You could, for example, buy an instrument that pays you a set amount for each degree, per day, that the temperature dips below the expected mean. If it stays warm, you lose; if a cold snap arrives, you win. It is, in essence, a form of weather-based insurance.

"Let's say you have a coat factory, like Burlington Coat Factory," Heppe says. "What happens if midway through the winter it's an El Niño year and it's very warm, and people do not purchase as many coats?" If it had purchased a put, it could have hedged its risk. In a warm winter, its weather future pays out and diminishes its coat-selling losses; in a cold winter, it loses the money on its future but makes it up on extra coat sales. "It's just like any natural hedge," he adds.

Heppe began selling weather futures ten years ago, and his main clients are energy firms, insurance companies, hedge funds, and investment banks, as well as some firms with explicit weather-related risk like ski lodges and resorts. The industry is small -- only about $16 billion a year -- because the idea still seems a bit weird.

"If you've got a business that sells $100 million a year, and you get up at the annual meeting and say, 'Hey, we should spend $10 million hedging against the risk of the weather,' it's hard to convince people," Heppe admits. "It's still a niche market. You have to educate people." But weather, he believes, is becoming a bigger area of public discussion, with the average executive increasingly well informed about it. "Technology is telling us there are recurring patterns we can spot," he says. "There's far more data than ever before." And the weather market isn't susceptible to being manipulated by powerful financial instruments, because the weather ultimately runs the show -- and nobody can control the weather. "You don't have somebody coming in and single- handedly driving the price of a commodity down to $100," he says with a laugh.

Has the new El Nino had any effect yet on his business? Heppe shakes his head. After watching a freak blizzard bury Buffalo in mid-October, he's betting El Niño stays weak for this winter season. But, as he admits, there's always the outside chance that anything could happen: "It's the weather."


Your five-year weather forecast.

Nobody really knows what'll happen more than a week in advance, of course. But if we assemble these major climatic trends, a rough snapshot of New York's future begins to emerge.

First off, El Nino will keep our winters reasonably mild and reduce hurricanes in the immediate future, possibly until as late as 2008, because El Ninos usually last for only one or two years.

Meanwhile, the AMO will remain in its warm phase, charging up storms and hurricanes off our shores, for much longer, probably another twenty years. So while El Nino may be driving a temporary reprieve in our nasty weather, once it dissipates, the long-term trend is back to tumultuous hurricane seasons.

The final ingredient in the mix is global warming. In the past century, the average temperature in New York has risen by two degrees, and the trend shows no sign of slowing down. Indeed, the computer models reviewed in the "Metropolitan East Coast Climate Assessment" -- a 50-year prediction of New York's changing climate, developed by NASA and Columbia University -- suggest that the city will continue to heat up by as much as one degree by 2010, two degrees by 2020, and accelerate on a gentle curve until we reach as much as nine degrees warmer than now in 2100. It doesn't particularly matter whether you believe the warming is man-made or a natural cycle (most, but not all, climatologists believe the former). The point is, pumping that much extra energy into the system is bound to have some effect.

The impact on our daily life, though, is the big question. A few degrees of warming won't turn New York into a Miami-class shirtsleeves town. The effect will be more subtle: Climate scientists suspect that a warmer climate will produce more weather volatility. It's not that we'll have more rain overall, more snow overall, or more storms overall. But each event will be more intense than before.

"We're more likely to get hotter heat waves," says Mark Cane, a climatologist at Columbia University. "And increased storminess" adds Cullen. Both effects are due to the additional energy that global warming pumps into the "hydrological cycle," the water and energy that circulates through the atmosphere -- and it's water that creates weather.

Indeed, we may already be seeing these effects. The last four winters have been seemingly pretty dry, but all have been punctuated by out-of-nowhere storms that dump so much snow on the city that the winter overall becomes wetter than normal. A string of hits like that hasn't happened since the late- nineteenth century, which climatologists find ominous. The virulence of recent hurricanes may also be another early effect. Last fall, a paper in Science pointed out that in the past 35 years, hurricanes have remained steady in sheer numbers -- there aren't more of them. But when they hit, they're more intense: The number of Category 1 hurricanes has shrunk, while the number of Category 4 and 5 hurricanes has doubled. Another paper in Nature found that hurricanes of the past 30 years have also increased in duration.

As Cullen puts it, global warming "messes with the probabilities. It loads the dice."

Fifty years from now, the city will be transformed by climate change. Our buildings will be greener, crafted with energy-saving local generators and solar panels; indeed, the latest crop of skyscrapers are already being built that way. Long Island coastline properties will increasingly look like a risky hurricane bet -- until everything calms down again in 2020, when the AMO shifts to cool. Spring will move a week or so earlier, as will the pollen season.

And, of course, the thick stands of ash in Central Park will vanish. But other things will emerge. The next time you look to the sky over Jamaica Bay, you may notice a string of long-necked birds flying in a peculiar formation -- a perfectly straight line. If you listen carefully, you might hear a cry that sounds like grrrrrr. It's the glossy ibis, a striking bird with a long, curved beak and a three-foot, shiny, bottle-green wingspan.

Five hundred years ago, local Indians would never have seen one of these birds. They're a southern species, normally more at home in the everglades of Florida or the Georgian marshes. But the city's already so much warmer that the Audubon chapter in New York has found that 300 mating pairs are now nesting in Jamaica Bay. "It's a really amazing sight," says E. J. McAdams, the group's former executive director. "They're really beautiful."

Enjoy them while you can -- because as the climate warms, the sea level off New York is predicted to rise by half a foot in the next fifteen years, swamping the ibis's new habitat in Jamaica Bay.

The weather giveth, and the weather taketh away.

Posted by Clive Thompson at 04:42 PM
Study: The better you know your spouse, the more likely you are to buy them a gift they hate









Here's a lovely, intriguing -- and, in keeping with the holiday spirit, totally depressing -- bit of holiday research: It turns out that the more familiar you are with your spouse or partner, the more likely you are to buy gifts they hate!

That's the result of an intriguing study recently done by Davy LeRouge and Luk Warlop (PDF link), two European professors of marketing. They took a few hundred couples, showed them pictures of furniture sets, and asked them to predict which ones their partners would like. Sometimes the people simply had to guess; in other situations, the experimenters would help them out by telling them which furniture settings their partners themselves had picked. And then the researchers also asked the subjects to predict which furniture settings would please a total stranger.

The result? People were pretty good at predicting the likes and dislikes of total strangers -- yet astonishly crappy at figuring out the preferences of their closest and dearest partners. And when they were given new information about which furniture their partners picked for themselves? It didn't help. They were still better at picking gifts for total strangers than for their loved ones.

Why? Possibly, the professors theorized, because when we're very familiar with our spouses it can be hard to separate our own preferences from theirs. We mistake things we'd like for things they'd like. Also, we tend to cherish hidebound ideas about what our partners are like, and we're unable to step outside those assumptions -- even when our partners themselves give us fresh, new information. When we face down strangers, we have none of those biases and thus are able to more clearly see them as they are.

As LeRouge and Warlop write:

Our findings reveal that being familiar with the person for whom you are predicting the product attitudes is a burden rather than an advantage, mainly because it prevents people from taking full advantage of newly provided information about that person's product attitudes ... Further evidence shows that it is the extensive amount of vivid information that the predictors hold about familiar others that prevents them from selecting more valid prediction cues.

Deck the halls. Mind you, maybe these sorts of errors are propping up the world economy. Think of it this way: What you really want is a new pair of jeans ... but instead, your partner thoughfully buys you a bunch of CDs of bands that he likes and you loathe. So you go out after the holidays and buy the jeans yourself. Essentially, there have been two rounds of gift-buying: Your partner's addled, narcissistic purchase of crap you don't want, and your own purchase of things you'd actually like. Double the spending -- double the boost to the economy! And double the landfill!

Posted by Clive Thompson at 03:35 PM
$2,000: The price of getting a PS3 four days before anyone else








Speaking of totally insane Xmas shopping binges -- I've been intrigued to watch the evolution of the pricing of the Playstation 3 on Ebay. Since Sony produced way less PS3's than people wanted, the auction market has been going crazy with the devices changing hands. And this means, of course, that we can accurately measure the outer limit of what people are willing to pay to get their hands on a PS3.

I can't claim to be tracking this with any scientific rigor, but from occasionally dropping into Ebay and checking and the prices for PS3s in the weeks directly after the release of the device, this is the trend that I noticed:

- Nov 17, the day the PS3 was released: $3,000
- four days later: $1,000
- one month later: $950

What this means is the truly hard-core early-adopters were willing to pay a premium of $2,000 just so they could get their hands on a PS3 four days earlier than anyone else. Then the market quickly settled on $1,000 as a reasonable price to pay for a PS3, without having to either wait in massive lineups at a game store.

Posted by Clive Thompson at 03:24 PM
New newsgame: Xtreme Xmas Shopping









My friend Ian Bogost -- and his insanely talented group of designers at Persuasive Games -- have just released their latest newsgame: Xtreme Xmas Shopping. As they describe it:

Only in America can shopping be considered a contact sport. And Christmas is the Super Bowl for competitive consumers. You've got a list of must-buy toys for your little toddler, and you'll be damned if someone else gets those gifts before you! Use whatever means necessary (physical harm?) to snatch this season's hottest toys before the other greedy shoppers get their hands on them!

Oh -- and merry Christmas!!

It's actually quite fun! I confess that I've always enjoyed the spectacle of toy crazes that produce Australian-rules, smashmouth holiday shopping. Back when Tickle Me Elmo first came out and kids went nonlinear over it, there was an incident in Fredericton, New Brunswick, in which a Wal-Mart employee was sent to the hospital after being stampeded by rabid parents. (Up in Canada, newscasters started referring to the doll as "Trample Me Elmo".) If there actually were a "War on Christmas", which there isn't, one could scarcely ask for a better culprit than America's annual, Caligulan orgy of consumption.

Posted by Clive Thompson at 02:52 PM
Comments working again, again

After breaking my blogging-fast last week, I awoke to discover that my commenting system was b0rked yet again. This time, apparently what happened was spambots pounded my comment script so ferociously -- 13,000 calls in one day -- that my hosting service, Pair.com, shut down all access to the script, so that not even I had the ability to restore the permissions. The long story is that I had to update to the latest version of Movable Type and install a plugin called AutoBan that attempts to help prevent flood attacks.

When I was young, did my mother sit me on her knee and say, "Son, one day you'll spend two entire workdays revamping your blog's code so that you can thwart spambot flood attacks?" No, she did not. Yet here we are, sigh.

Anyway, comments are up and working again!

Posted by Clive Thompson at 02:42 PM
December 05, 2006
Why interactive websites can create false memories










Can an interactive web site produce false memories?

Possibly so, according to a fascinating paper to be published this month in the Journal of Consumer Research by Ann Schlosser, a business professor at the University of Washington. Schlosser performed an intriguing experiment: She took two groups of people and had them check out two different web sites devoted to the same digital camera. One site included static pictures; the other was interactive, allowing users to play around with a virtual version of the product.

Later, she tested them on their ability to recall details about the camera. She intentionally included details that were false, but sufficiently plausible that they might have been true. The result? The people who viewed the interactive demo of the camera were much more likely than the folks who'd only viewed static images to "remember" the false details as being present. Or another way of putting it: The interactive demo was more likely to produce false memories of the product -- potential buyers who thought the camera could do things it can't.

Why? Schlosser theorizees that it's partly because interactivity encourages more "certainty" in our memories, and thus increases the likelihood that we'll believe suggestively false details to be true. And, as she concludes:

These findings suggest that marketing managers should test their campaigns for both true and false memories. Although it may seem advantageous for consumers to believe that a product has features that it actually does not have (e.g., by increasing store visits and purchases), it may ultimately lead to customer dissatisfaction. Because false memories reflect source-monitoring errors—or believing that absent attributes were actually presented in the marketing campaign—consumers who discover that the product does not have these attributes will likely feel misled by the company.

One interesting thing Schlosser points out is that market-research folks almost never study the false-memory effects of advertising. Sure, they test to see whether consumers who've looked at promotional material can recall true information about a product. But they rarely check to see whether the consumers also remember false information. An interesting -- if telling -- elision, eh?

This also makes me wonder about whether other virtual-reality environments, such as simulation video games, can create false pools of knowledge. This is a potentially a big deal for the "serious games" folks, because many of them create brilliant little simulations as a way of educating people about complex situations. Cool enough! But what if they these sims also unintentionally impart bogus knowledge -- making the gamers feel so artificially sure of the complex system that they attribute properties to it that don't exist?

Interesting stuff to think about, either way.

Posted by Clive Thompson at 10:43 PM
"Playce" -- a website you navigate by playing video games











As I've written before, video games were the first place we learned how to interact with information on a digital screen. Icons? Controller movement? Screen-scrolling? Navigation of complex menus? All these concepts now part of computer interface design were first hacked out in games.

So now Steffen Walz, a PhD student at the Swiss Federal Institute of Technology, has closed this loop -- by designing "Playce", a web site that you navigate by playing it like a retro-80s video game.

Playce is divided into two panels vertically, and you begin by picking one of four different types of games based on your personality. As you play the game, you unlock different parts of Walz's site (which is mostly a portfolio of his design work). Pick the "achiever" game and you'll be playing a version of Breakout, where you have to destroy certain bricks to navigate to different site pages. Pick the "killer" game, and you play an old-skool shooter where you blast little tanks, soldiers or planes to go to pages. (That's a snapshot of one section of the "killer" screen above.)

As Walz notes on his web site:

The art and craft of make-believe place-making challenges architects, urban planners, game and interaction designers, and it likely to (need to) take advantages not only of the game generation's competencies ... but also reflect the expectations of the Homo Ludens Digitalis, who has been trained to win not only in the gamespace, but in the gamespace that is everyday.

As Walz noted in an email to me, the interface isn't exactly an efficient way to navigate, but it's pretty thought-provoking. And it makes me wonder: Are there any examples out there of web sites that navigate in gamelike fashions, without directly referencing games? I.e. in a more subconscious way?

Posted by Clive Thompson at 10:04 PM
December 04, 2006
The flip-book of your life









What would it be like to view your entire life in a few minutes? Last month, I wrote a Fast Company article that talks about Gordon Bell's attempt to record everything that happens to him. One of the things he uses is a Microsoft SenseCam -- an experimental, wearable camera that automatically snaps pictures of what you're looking at, all day long. The question is, what do you do with all those zillions of pictures? Is there any way to use them to improve your memory or cognition?

Well, as I noted in the story, a couple of Irish and British scientists tried something interesting: At the end of each day, they'd download the day's pictures and quickly scroll through them like a flashbook -- viewing hundreds of snaps in a minute or so. They discovered that it would help "seal" the day's events in your real, brain-based memory. (Indeed, it even drastically improved the everyday recall of a woman who suffers from ongoing amnesia.)

William Braine, a friend of mine, read my article and then had his own experience of this effect -- inadvertantly. As he wrote in an instant message to me:

This weekend I transferred the contents of two older computers to my new iMac. When I imported the 3000-or-so photos from 1998-2006 into the new machine, they flashed by at about a quarter-second each. I got to see shots of our honeymoon, our apartment, a fat me, an ultrasound, a thin me, a newborn, a new house, a baby, another new house -- with vacations and friends and family all speeding through ... amazing.

Cool, eh? Since so many people now snap tons of pictures of their daily activities, I'd imagine there's a good market for simple screensaver-like apps that intelligently sort your pictures and then whizz through them in different ways, to produce this sort of cognitive priming. And the most interesting effects aren't necessarily about remembering things in a utilitarian way; they're probably more about, as Bill noted above, the emotional aspect -- different ways of re-experiencing and assessing your life.

Imagine being 60 years old, and having one psychologically significant picture taken from each month of an entire life's archive. That's 720 photos. Scroll them by at the speed that Bill experienced -- four per second -- and your life would flash by in three minutes. What in god's name would that feel like? I figure whatever version of Flickr that exists 50 years from now will have this sort of capability, so I guess I'll eventually find out.

Posted by Clive Thompson at 04:34 PM
City life speeds up birdsong









People love to joke about how big-city life makes people talk faster, walk faster, and generally stress out. But according to a study in this month's issue of Current Biology, the urban environment also turbocharges birds. A couple of Dutch scientists recorded the song of the Great Tit -- Parus major, pictured above -- in both forest settings and city settings, then compared them. The result? As they note in their abstract:

Urban songs were shorter and sung faster than songs in forests, and often concerned atypical song types. Furthermore, we found consistently higher minimum frequencies in ten out of ten city-forest comparisons from London to Prague and from Amsterdam to Paris.

Their paper is behind a paywall, alas, but for free they put online a dozen audio samples of the bird songs -- so you can hear the difference for yourself! It's pretty cool. Go to the bottom of the page and listen in particular to the comparison of the songs recorded in downtown Paris versus the rural area of Fontainebleu; it's precisely the same call, except pitchshifted higher and faster in Paris.

Evolutionary scientists have theorized for decades that the noisy environment of cities would alter birdcall, but this is some of the most impressive evidence yet assembled. It's also metaphorically lovely. When I spill my coffee while fumbling to answer a mobile-phone call while racing for the subway next week, I can take comfort: Even the songbirds are racing to keep up.


(Thanks to Eurekalert for this one!)

Posted by Clive Thompson at 04:09 PM
The sonnetry of Gears of War: My latest Wired News gaming column









When is a gore-soaked video game like a Shakespearean sonnet? When it's Gears of War, my friends. Or so I argue in my latest Wired News column. Check it out online free at the Wired News site, or archived below, and see if you agree!

Why Gears of War rocks
by Clive Thompson

Why is Gears of War so insanely awesome?

Technically, it's just the same old, same old. You've seen stuff like this a bazillion times before. Gears of War is yet another run-and-gun shooter in which you blunder through the post-apocalyptic boneyard of civilization, repetitively slaughtering a bunch of hulking, gibbering aliens. Creepy things lurk in the dark; fresh ammo packs are scattered improbably in open sight; and as the guts paint the hallways red, your teammates curse like a bunch of Tarantino wannabes. Name every single war-weary cliche of the run-and-gun genre, and Gears of War dutifully ticks it off.

Yet the game really is awesome. Indeed, it is staggeringly, derangedly so. I popped Gears of War into my Xbox 360 and sat in a cybernetic haze for three straight hours, emerging with my stomach in fist-size knots, so emotionally and cognitively depleted that I had to consult the instructions on the side of the box before I was able to cook a bag of microwave popcorn -- which, come to think of it, was my only meal for the rest of the evening because I had to go back and play until I collapsed.

Normally, I am the first guy to complain about the lack of creativity in today's games. I've argued many times that games are being held back by publishers who refuse to experiment -- and insist on sticking to the same five or six numbingly familiar genres. Shouldn't we be breaking ground with risky new forms of play? Do we really need yet another run-and-gun shooter?

Well, Gears of War convinces me that jeez, maybe we do. That's because creativity does not come only from a daring, new art form or weird new genre. It also comes from a dog-eared, well-worn genre that is proven to work -- and is constantly tweaked by artists who love it.

Consider the sonnet. It's been around ever since Italian poets invented it in the 13th century, and it's deeply formulaic. But it's never gotten boring, because poets keep on finding surprising new ways to hack it. The Earl of Surrey remixed the sonnet's 14 lines into a new stanzaic structure, turning it into a four-part argument and spurring Shakespeare into an orgy of creativity. Then e. e. cummings tore the sonnet into tiny shreds, splaying the words across the page while using the rhyming structure to hold each poem together.

A more modern example is the three-minute pop song. The verse-chorus-verse structure is as repetitive as you can get. But for music fans, part of the fun is wondering how a band will do something unique and fresh with it. Lame songs fail to surprise; superb ones somehow manage to push the cultural glacier forward an inch or two. The point is, the music relies on our longstanding familiarity with its tropes.

Thus it is with Gears of War. Every element is simultaneously totally familiar and a bit surprising. Sure, you have to dodge enemy fire, just like every shooter in history. But the mechanics of hiding behind objects are executed with iPod-like elegance. A single button lets you feint from object to object, and a single trigger lets you pop out to fire off a shot before ducking back again. The ease of dodging transforms each rubble-strewn scene into a spatial puzzle: What can I hide behind? Where can I scootch over to get a better shot?

Virtually every element in the game has been similarly torqued. You know how the aliens in most shooters always sound like the squealing demon-pig noises of The Exorcist? Well, that's precisely how the Berserker sounds in Gears of War ... except that the audio engineers have somehow produced an acoustic atrocity that scrapes like Satan's own fingernails across the blackboard of your mind. I pretty much wet myself.

I could go on. The camera work is nifty: When you break into a tuck-and-run military dash to scurry across an open plain, the camera zooms out low beside you, as if you were being tracked by a panicked CNN videographer. And there's a machine gun with a chainsaw on the end, which transforms the gun into a clever, metaphoric gloss on the age-old bayonet ... as well as, y'know, transforming it into a gun with a chainsaw on the end. Hell yes. It's the little things, people, the little things!

So forgive me if I suspend, for a few weeks, my strident insistence that games break new ground, innovate new forms, revolutionize the nature of play. Gears of War doesn't, and it doesn't need to. It's the same old, same old -- made wonderfully new.

Posted by Clive Thompson at 02:48 PM
Octopus yoga








This one has already been all over Digg and Boing Boing, but I can't resist. Behold: An octopus escaping from an underground box by squeezing its entire body through a teensy one-inch hole.

As I've been warning for years -- as soon as these guys get tired of our crap, we are doomed. Why hasn't someone made a movie about octopuses rising up from the briny deepy, penetrating our trillion-dollar anti-celaphopod national defenses by squeezing under door-cracks, and just totally handing Western civilization its ass? Possibly because it would be too close to the terrifying truth.

Posted by Clive Thompson at 02:43 PM
Why meatspace is where it's at









Steven Johnson just published The Ghost Map, a superb book about the 1854 outbreak of cholera in London -- and how its cause was uncovered by a clergyman and a doctor who used local maps to grok the topology of the outbreak. Cool enough, but Johnson used his thinking about neighborhoods and mapping to create a new website called outside.in.

It's a pretty simple concept: You type in your zip code or address, and outside.in shows you any relevant info online -- ranging from blogger reviews about nearby restaurants to tidbits in local papers. I popped in my zip code -- 10011 -- and got info about a new nearby Austrian restaurant, city-council tax reform, and nearby artists working on a Darfur project.

The really interesting thing here, though, is Johnson's philosophy behind the project: The seemingly paradoxical proposition that while Internet technologies were originally touted as "making geography irrelevant", in actual fact they excel at the opposite -- giving you richer info about the stuff that's going on nearby you. As Johnson told today's New York Times Arts section:

"It really shows that the old idea that the Internet was going to make cities obsolete had it exactly wrong," he said. "In fact the Internet enhances cities in all these different ways. I think it lets people have the kinds of conversations that we sentimentally always imagined that people were having."

"When you combine that mix of the opportunity for discussion and debate between people who don't necessarily know each other, when it's all grounded in an actual physical place and it's not just about going into a game world and arguing over dragons or something like that," he continued, "then I think you have something that is a real enhancement of civic conversation and the kind of public space that's so important in a great city."

Amen. Much as I thrive in virtual worlds -- from World of Warcraft to the blogosphere to ECHO -- you can't deny that meatspace is where it's at. Mind you, if I didn't already love the idea of being surrounded by millions of interesting strangers and having their lives collide with mine at unpredictable moments and with a wildly varying quality of results, I wouldn't live in New York. Heh.

Posted by Clive Thompson at 02:02 PM
Open-source spying: My latest piece for the New York Times Magazine









Yesterday, the New York Times Magazine published a piece I wrote about whether Web 2.0 technologies like wikis and blogs could help improve the dot-connecting ability of the US' intelligence agencies. The story is online at the web site, linked above, but here's a permanent copy too below!

(That image above, by the way, is from the excellent graphics done for the piece by Lisa Strausfeld and James Nick Sears, illustrating networked connections between various terrorism-related terms.)

Open-Source Spying
Could blogs and wikis help stop the next 9/11?
by Clive Thompson

When Matthew Burton arrived at the Defense Intelligence Agency in January 2003, he was excited about getting to his computer. Burton, who was then 22, had long been interested in international relations: he had studied Russian politics and interned at the U.S. consulate in Ukraine, helping to speed refugee applications of politically persecuted Ukrainians. But he was also a big high-tech geek fluent in Web-page engineering, and he spent hours every day chatting online with friends and updating his own blog. When he was hired by the D.I.A., he told me recently, his mind boggled at the futuristic, secret spy technology he would get to play with: search engines that can read minds, he figured. Desktop video conferencing with colleagues around the world. If the everyday Internet was so awesome, just imagine how much better the spy tools would be.

But when he got to his cubicle, his high-tech dreams collapsed. ''The reality,'' he later wrote ruefully, ''was a colossal letdown.''

The spy agencies were saddled with technology that might have seemed cutting edge in 1995. When he went onto Intelink -- the spy agencies' secure internal computer network -- the search engines were a pale shadow of Google, flooding him with thousands of useless results. If Burton wanted to find an expert to answer a question, the personnel directories were of no help. Worse, instant messaging with colleagues, his favorite way to hack out a problem, was impossible: every three-letter agency -- from the Central Intelligence Agency to the National Security Agency to army commands -- used different discussion groups and chat applications that couldn't connect to one another. In a community of secret agents supposedly devoted to quickly amassing information, nobody had even a simple blog -- that ubiquitous tool for broadly distributing your thoughts.

Something had gone horribly awry, Burton realized. Theoretically, the intelligence world ought to revolve around information sharing. If F.B.I. agents discover that Al Qaeda fund-raising is going on in Brooklyn, C.I.A. agents in Europe ought to be able to know that instantly. The Internet flourished under the credo that information wants to be free; the agencies, however, had created their online networks specifically to keep secrets safe, locked away so only a few could see them. This control over the flow of information, as the 9/11 Commission noted in its final report, was a crucial reason American intelligence agencies failed to prevent those attacks. All the clues were there -- Al Qaeda associates studying aviation in Arizona, the flight student Zacarias Moussaoui arrested in Minnesota, surveillance of a Qaeda plotting session in Malaysia -- but none of the agents knew about the existence of the other evidence. The report concluded that the agencies failed to ''connect the dots.''

By way of contrast, every night when Burton went home, he was reminded of how good the everyday Internet had become at connecting dots. ''Web 2.0'' technologies that encourage people to share information -- blogs, photo-posting sites like Flickr or the reader-generated encyclopedia Wikipedia -- often made it easier to collaborate with others. When the Orange Revolution erupted in Ukraine in late 2004, Burton went to Technorati, a search engine that scours the ''blogosphere,'' to find the most authoritative blog postings on the subject. Within minutes, he had found sites with insightful commentary from American expatriates who were talking to locals in Kiev and on-the-fly debates among political analysts over what it meant. Because he and his fellow spies were stuck with outdated technology, they had no comparable way to cooperate -- to find colleagues with common interests and brainstorm online.

Burton, who has since left the D.I.A., is not alone in his concern. Indeed, throughout the intelligence community, spies are beginning to wonder why their technology has fallen so far behind -- and talk among themselves about how to catch up. Some of the country's most senior intelligence thinkers have joined the discussion, and surprisingly, many of them believe the answer may lie in the interactive tools the world's teenagers are using to pass around YouTube videos and bicker online about their favorite bands. Billions of dollars' worth of ultrasecret data networks couldn't help spies piece together the clues to the worst terrorist plot ever. So perhaps, they argue, it' s time to try something radically different. Could blogs and wikis prevent the next 9/11?


THE JOB OF AN ANALYST used to be much more stable -- even sedate. In the '70s and '80s, during the cold war, an intelligence analyst would show up for work at the C.I.A.'s headquarters in Langley, Va., or at the National Security Agency compound in Fort Meade, Md., and face a mess of paper. All day long, tips, memos and reports from field agents would arrive: cables from a covert-ops spy in Moscow describing a secret Soviet meeting, or perhaps fresh pictures of a missile silo. An analyst's job was to take these raw pieces of intelligence and find patterns in the noise. In a crisis, his superiors might need a quick explanation of current events to pass on to their agency heads or to Congress. But mostly he was expected to perform long-term ''strategic analysis'' -- to detect entirely new threats that were still forming.

And during the cold war, threats formed slowly. The Soviet Union was a ponderous bureaucracy that moved at the glacial speed of the five-year plan. Analysts studied the emergence of new tanks and missiles, pieces of hardware that took years to develop. One year, an analyst might report that the keel for a Soviet nuclear submarine had been laid; a few years later, a follow-up report would describe the submarine's completion; even more years later, a final report would detail the sea trials. Writing reports was thus a leisurely affair, taking weeks or months; thousands of copies were printed up and distributed via interoffice mail. If an analyst's report impressed his superiors, they'd pass it on to their superiors, and they to theirs -- until, if the analyst was very lucky, it landed eventually in the president's inner circle. But this sort of career achievement was rare. Of the thousands of analyst reports produced each year, the majority sat quietly gathering dust on agency shelves, unread by anyone.

Analysts also did not worry about anything other than their corners of the world. Russia experts focused on Russia, Nicaragua ones on Nicaragua. Even after the cold war ended, the major spy agencies divided up the world: the F.B.I. analyzed domestic crime, the C.I.A. collected intelligence internationally and military spy agencies, like the National Security Agency and National Geospatial-Intelligence Agency, evaluated threats to the national defense. If an analyst requested information from another agency, that request traveled through elaborate formal channels. The walls between the agencies were partly a matter of law. The charters of the C.I.A. and the defense intelligence agencies prohibited them from spying on American citizens, under the logic that the intrusive tactics needed to investigate foreign threats would violate constitutional rights if applied at home. The F.B.I. even had an internal separation: agents investigating terrorist activity would not share information with those investigating crimes, worried that secrets gleaned from tailing Al Qaeda operatives might wind up publicly exposed in a criminal trial.

Then on Sept. 12, 2001, analysts showed up at their desks and faced a radically altered job. Islamist terrorists, as 9/11 proved, behaved utterly unlike the Soviet Union. They were rapid-moving, transnational and cellular. A corner-store burglar in L.A. might turn out to be a Qaeda sympathizer raising money for a plot being organized overseas. An imam in suburban Detroit could be recruiting local youths to send to the Sudan for paramilitary training. Al Qaeda operatives organized their plots in a hivelike fashion, with collaborators from Afghanistan to London using e-mail, instant messaging and Yahoo groups; rarely did a single mastermind run the show. To disrupt these new plots, some intelligence officials concluded, American agents and analysts would need to cooperate just as fluidly -- trading tips quickly among agents and agencies. Following the usual chain of command could be fatal. ''To fight a network like Al Qaeda, you need to behave like a network,'' John Arquilla, the influential professor of defense at the Naval Postgraduate School, told me.

It was a fine vision. But analysts were saddled with technology that was designed in the cold war. They now at least had computers, and intelligence arrived as electronic messages instead of paper memos. But their computers still communicated almost exclusively with people inside their agencies. When the intelligence services were computerized in the '90s, they had digitally replicated their cold-war divisions -- each one building a multimillion-dollar system that allowed the agency to share information internally but not readily with anyone outside.

The computer systems were designed to be ''air gapped.'' The F.B.I. terminals were connected to one another -- but not to the computers at any other agency, and vice versa. Messages written on the C.I.A.'s network (which they still quaintly called ''cables'') were purely internal. To get a message to the F.B.I. required a special communication called a ''telegraphic dissemination.'' Each agency had databases to amass intelligence, but because of the air gap, other agencies could not easily search them. The divisions were partly because of turf battles and partly because of legal restrictions -- but they were also technological. Mike Scheuer, an adviser to the C.I.A.'s bin Laden unit until 2004, told me he had been frustrated by the inability of the systems to interpenetrate. ''About 80 percent of C.I.A.-F.B.I. difficulties came from the fact that we couldn't communicate with one another,'' he said. Scheuer told me he would often send a document electronically to the F.B.I., then call to make sure the agents got it. ''And they'd say, 'We can't find it, can you fax it?' And then we'd call, and they'd say, 'Well, the system said it came in, but we still can't find it -- so could you courier it over?' '' ''


"THESE SYSTEMS HAVE SERVED US VERY WELL for five decades,'' Dale Meyerrose told me when I spoke with him recently. But now, he said, they're getting in the way. ''The 16 intelligence organizations of the U.S. are without peer. They are the best in the world. The trick is, are they collectively the best?''

Last year, Meyerrose, a retired Air Force major general, was named the chief information officer -- the head computer guy, as it were -- for the office of the director of national intelligence. Established by Congress in 2004, the D.N.I.'s office has a controversial mandate: it is supposed to report threats to the president and persuade the intelligence agencies to cooperate more closely. Both tasks were formerly the role of the C.I.A. director, but since the C.I.A. director had no budgetary power over the other agencies, they rarely heeded his calls to pass along their secrets. So the new elevated position of national-intelligence director was created; ever since, it has been filled by John Negroponte. Last December, Negroponte hired Meyerrose and gave him the daunting task of developing mechanisms to allow the various agencies' aging and incompatible systems to swap data. Right away, Meyerrose ordered some sweeping changes. In the past, each agency chose its own outside contractor to build customized software -- creating proprietary systems, each of which stored data in totally different file formats. From now on, Meyerrose said, each agency would have to build new systems using cheaper, off-the-shelf software so they all would be compatible. But bureaucratic obstacles were just a part of the problem Meyerrose faced. He was also up against something deeper in the DNA of the intelligence services. ''We've had this 'need to know' culture for years,'' Meyerrose said. ''Well, we need to move to a 'need to share' philosophy.''

There was already one digital pipeline that joined the agencies (though it had its own limitations): Intelink, which connects most offices in each intelligence agency. It was created in 1994 after C.I.A. officials saw how the Web was rapidly transforming the way private-sector companies shared information. Intelink allows any agency to publish a Web page, or put a document or a database online, secure in the knowledge that while other agents and analysts can access it, the outside world cannot.

So why hasn't Intelink given young analysts instant access to all secrets from every agency? Because each agency's databases, and the messages flowing through their internal pipelines, are not automatically put onto Intelink. Agency supervisors must actively decide what data they will publish on the network -- and their levels of openness vary. Some departments have created slick, professional sites packed full of daily alerts and searchable collections of their reports going back years. Others have put up little more than a ''splash page'' announcing they exist. Operational information -- like details of a current covert action -- is rarely posted, usually because supervisors fear that a leak could jeopardize a delicate mission.

Nonetheless, Intelink has grown to the point that it contains thousands of agency sites and several hundred databases. Analysts at the various agencies generate 50,000 official reports a year, many of which are posted to the network. The volume of material online is such that analysts now face a new problem: data overload. Even if they suspect good information might exist on Intelink, it is often impossible to find it. The system is poorly indexed, and its internal search tools perform like the pre-Google search engines of the '90s.''

One of my daily searches is for words like 'Afghanistan' or 'Taliban,' '' I was told by one young military analyst who specializes in threats from weapons of mass destruction. (He requested anonymity because he isn't authorized to speak to reporters.) ''So I'm looking for reports from field agents saying stuff like, 'I'm out here, and here's what I saw,' '' he continued. ''But I get to my desk and I've got, like, thousands a day -- mountains of information, and no way to organize it.''

Adding to the information glut, there's an increasingly large amount of data to read outside of Intelink. Intelligence analysts are finding it more important to keep up with ''open source'' information -- nonclassified material published in full public view, like newspapers, jihadist blogs and discussion boards in foreign countries. This adds ever more calories to the daily info diet. The W.M.D. analyst I spoke to regularly reads the blog of Juan Cole, a University of Michigan professor known for omnivorous linking to, and acerbic analysis of, news from the Middle East. ''He's not someone spies would normally pay attention to, but now he's out there -- and he's a subject-matter expert, right?'' the analyst said.

Intelligence hoarding presented one set of problems, but pouring it into a common ocean, Meyerrose realized soon after moving into his office, is not the answer either. ''Intelligence is about looking for needles in haystacks, and we can't just keep putting more hay on the stack,'' he said. What the agencies needed was a way to take the thousands of disparate, unorganized pieces of intel they generate every day and somehow divine which are the most important.


INTELLIGENCE HEADS WANTED TO TRY to find some new answers to this problem. So the C.I.A. set up a competition, later taken over by the D.N.I., called the Galileo Awards: any employee at any intelligence agency could submit an essay describing a new idea to improve information sharing, and the best ones would win a prize. The first essay selected was by Calvin Andrus, chief technology officer of the Center for Mission Innovation at the C.I.A. In his essay, ''The Wiki and the Blog: Toward a Complex Adaptive Intelligence Community,'' Andrus posed a deceptively simple question: How did the Internet become so useful in helping people find information?

Andrus argued that the real power of the Internet comes from the boom in self-publishing: everyday people surging online to impart their thoughts and views. He was particularly intrigued by Wikipedia, the ''reader-authored'' encyclopedia, where anyone can edit an entry or create a new one without seeking permission from Wikipedia's owners. This open-door policy, as Andrus noted, allows Wikipedia to cover new subjects quickly. The day of the London terrorist bombings, Andrus visited Wikipedia and noticed that barely minutes after the attacks, someone had posted a page describing them. Over the next hour, other contributors -- some physically in London, with access to on-the-spot details -- began adding more information and correcting inaccurate news reports. ''You could just sit there and hit refresh, refresh, refresh, and get a sort of ticker-tape experience,'' Andrus told me. What most impressed Andrus was Wikipedia's self-governing nature. No central editor decreed what subjects would be covered. Individuals simply wrote pages on subjects that interested them -- and then like-minded readers would add new facts or fix errors. Blogs, Andrus noted, had the same effect: they leveraged the wisdom of the crowd. When a blogger finds an interesting tidbit of news, he posts a link to it, along with a bit of commentary. Then other bloggers find that link and, if they agree it's an interesting news item, post their own links pointing to it. This produces a cascade effect. Whatever the first blogger pointed toward can quickly amass so many links pointing in its direction that it rockets to worldwide notoriety in a matter of hours.

Spies, Andrus theorized, could take advantage of these rapid, self-organizing effects. If analysts and agents were encouraged to post personal blogs and wikis on Intelink -- linking to their favorite analyst reports or the news bulletins they considered important -- then mob intelligence would take over. In the traditional cold-war spy bureaucracy, an analyst's report lived or died by the whims of the hierarchy. If he was in the right place on the totem pole, his report on Soviet missiles could be pushed up higher; if a supervisor chose to ignore it, the report essentially vanished. Blogs and wikis, in contrast, work democratically. Pieces of intel would receive attention merely because other analysts found them interesting. This grass-roots process, Andrus argued, suited the modern intelligence challenge of sifting through thousands of disparate clues: if a fact or observation struck a chord with enough analysts, it would snowball into popularity, no matter what their supervisors thought.

A profusion of spy blogs and wikis would have another, perhaps even more beneficial impact. It would drastically improve the search engines of Intelink. In a paper that won an honorable mention in the Galileo Awards, Matthew Burton -- the young former D.I.A. analyst -- made this case. He pointed out that the best Internet search engines, including Google, all use ''link analysis'' to measure the authority of documents. When you type the search ''Afghanistan'' into Google, it finds every page that includes that word. Then it ranks the pages in part by how many links point to the page -- based on the idea that if many bloggers and sites have linked to a page, it must be more useful than others. To do its job well, Google relies on the links that millions of individuals post online every day.

This, Burton pointed out, is precisely the problem with Intelink. It has no links, no social information to help sort out which intel is significant and which isn't. When an analyst's report is posted online, it does not include links to other reports, even ones it cites. There's no easy way for agents to link to a report or post a comment about it. Searching Intelink thus resembles searching the Internet before blogs and Google came along -- a lot of disconnected information, hard to sort through. If spies were encouraged to blog on Intelink, Burton reasoned, their profuse linking could mend that situation. ''

Imagine having tools that could spot emerging patterns for you and guide you to documents that might be the missing pieces of evidence you're looking for,'' Burton wrote in his Galileo paper. ''Analytical puzzles, like terror plots, are often too piecemeal for individual brains to put together. Having our documents aware of each other would be like hooking several brains up in a line, so that each one knows what the others know, making the puzzle much easier to solve.''

With Andrus and Burton's vision in mind, you can almost imagine how 9/11 might have played out differently. In Phoenix, the F.B.I. agent Kenneth Williams might have blogged his memo noting that Al Qaeda members were engaging in flight-training activity. The agents observing a Qaeda planning conference in Malaysia could have mentioned the attendance of a Saudi named Khalid al-Midhar; another agent might have added that he held a multi-entry American visa. The F.B.I. agents who snared Zacarias Moussaoui in Minnesota might have written about their arrest of a flight student with violent tendencies. Other agents and analysts who were regular readers of these blogs would have found the material interesting, linked to it, pointed out connections or perhaps entered snippets of it into a wiki page discussing this new trend of young men from the Middle East enrolling in pilot training.

As those four original clues collected more links pointing toward them, they would have amassed more and more authority in the Intelink search engine. Any analysts doing searches for ''Moussaoui'' or ''Al Qaeda'' or even ''flight training'' would have found them. Indeed, the original agents would have been considerably more likely to learn of one another's existence and perhaps to piece together the topography of the 9/11 plot. No one was able to prevent 9/11 because nobody connected the dots. But in a system like this, as Andrus's theory goes, the dots are inexorably drawn together. ''Once the intelligence community has a robust and mature wiki and blog knowledge-sharing Web space,'' Andrus concluded in his essay, ''the nature of intelligence will change forever.''


AT FIRST GLANCE, THE IDEA might seem slightly crazy. Outfit the C.I.A. and the F.B.I. with blogs and wikis? In the civilian world, after all, these online tools have not always amassed the most stellar reputations. There are many valuable blogs and wikis, of course, but they are vastly outnumbered by ones that exist to compile useless ephemera, celebrity gossip and flatly unverifiable assertions. Nonetheless, Andrus's ideas struck a chord with many very senior members of the office of the director of national intelligence. This fall, I met with two of them: Thomas Fingar, the patrician head of analysis for the D.N.I., and Mike Wertheimer, his chief technology officer, whose badge clip sports a button that reads ''geek.'' If it is Meyerrose's job to coax spy hardware to cooperate, it is Fingar's job to do the same for analysts.

Fingar and Wertheimer are now testing whether a wiki could indeed help analysts do their job. In the fall of 2005, they joined forces with C.I.A. wiki experts to build a prototype of something called Intellipedia, a wiki that any intelligence employee with classified clearance could read and contribute to. To kick-start the content, C.I.A. analysts seeded it with hundreds of articles from nonclassified documents like the C.I.A. World Fact Book. In April, they sent out e-mail to other analysts inviting them to contribute, and sat back to see what happened.

By this fall, more than 3,600 members of the intelligence services had contributed a total of 28,000 pages. Chris Rasmussen, a 31-year-old ''knowledge management'' engineer at the National Geospatial-Intelligence Agency, spends part of every day writing or editing pages. Rasmussen is part of the younger generation in the intelligence establishment that is completely comfortable online; he regularly logs into a sprawling, 50-person chat room with other Intellipedians, and he also blogs about his daily work for all other spies to read. He told me the usefulness of Intellipedia proved itself just a couple of months ago, when a small two-seater plane crashed into a Manhattan building. An analyst created a page within 20 minutes, and over the next two hours it was edited 80 times by employees of nine different spy agencies, as news trickled out. Together, they rapidly concluded the crash was not a terrorist act. ''In the intelligence community, there are so many 'Stay off the grass' signs,'' Rasmussen said. ''But here, you're free to do what you want, and it works.''

By the late summer, Fingar decided the Intellipedia experiment was sufficiently successful that he would embark on an even more high-profile project: using Intellipedia to produce a ''national intelligence estimate'' for Nigeria. An N.I.E. is an authoritative snapshot of what the intelligence community thinks about a particular state -- and a guide for foreign and military policy. Nigeria, Fingar said, is a complex country, with issues ranging from energy to Islamic radicalism to polio outbreaks to a coming election. Intellipedia's Nigeria page will harness the smarts of the dozen or so analysts who specialize in the country. But it will also, Fingar hopes, attract contributions from other intelligence employees who have expertise Fingar isn't yet aware of -- an analyst who served in the Peace Corps in Nigeria, or a staff member who has recently traveled there. In the traditional method of producing an intelligence estimate, Fingar said, he would call every agency and ask to borrow their Africa expert for a week or two of meetings. ''And they'd say: 'Well, I only got one guy who can spell Nigeria, and he's traveling. So you lose.' '' In contrast, a wiki will ''change the rules of who can play,'' Fingar said, since far-flung analysts and agents around the world could contribute, day or night.

Yet Intellipedia also courts the many dangers of wikis -- including the possibility of error. What's to stop analysts from posting assertions that turn out to be false? Fingar admits this will undoubtedly happen. But if there are enough people looking at an entry, he says, there will always be someone to catch any grave mistakes. Rasmussen notes that though there is often strong disagreement and debate on Intellipedia, it has not yet succumbed to the sort of vandalism that often plagues Wikipedia pages, including the posting of outright lies. This is partly because, unlike with Wikipedia, Intellipedia contributors are not anonymous. Whatever an analyst writes on Intellipedia can be traced to him. ''If you demonstrate you've got something to contribute, hey, the expectation is you're a valued member,'' Fingar said. ''You demonstrate you're an idiot, that becomes known, too.''

While the C.I.A. and Fingar's office set up their wiki, Meyerrose's office was dabbling in the other half of Andrus's equation. In July, his staff decided to create a test blog to collect intelligence. It would focus on spotting and predicting possible avian-flu outbreaks and function as part of a larger portal on the subject to collect information from hundreds of sources around the world, inside and outside of the intelligence agencies. Avian flu, Meyerrose reasoned, is a national-security problem uniquely suited to an online-community effort, because information about the danger is found all over the world. An agent in Southeast Asia might be the first to hear news of dangerous farming practices; a medical expert in Chicago could write a crucial paper on transmission that was never noticed by analysts.

In August, one of Meyerrose's assistants sat me down to show me a very brief glimpse of the results. In the months that it has been operational, the portal has amassed 38,000 ''active'' participants, though not everyone posts information. In one corner was the active-discussion area -- the group blog where the participants could post their latest thoughts about avian flu and others could reply and debate. I noticed a posting, written by a university academic, on whether the H5N1 virus could actually be transmitted to humans, which had provoked a dozen comments. ''See, these people would never have been talking before, and we certainly wouldn't have heard about it if they did,'' the assistant said. By September, the site had become so loaded with information and discussion that Rear Adm. Arthur Lawrence, a top official in the health department, told Meyerrose it had become the government's most crucial resource on avian flu.

The blog seemed like an awfully modest thing to me. But Meyerrose insists that the future of spying will be revolutionized as much by these small-bore projects as by billion-dollar high-tech systems. Indeed, he says that overly ambitious projects often result in expensive disasters, the way the F.B.I.'s $170 million attempt to overhaul its case-handling software died in 2005 after the software became so complex that the F.B.I. despaired of ever fixing the bugs and shelved it. In contrast, the blog software took only a day or two to get running. ''We need to think big, start small and scale fast,'' Meyerrose said.

Moving quickly, in fact, is crucial to building up the sort of critical mass necessary to make blogs and wikis succeed. Back in 2003, a Department of Defense agency decided to train its analysts in the use of blog software, in hopes that they would begin posting about their work, read one another's blogs and engage in productive conversations. But the agency's officials trained only small groups of perhaps five analysts a month. After they finished their training, those analysts would go online, excited, and start their blogs. But they'd quickly realize no one else was reading their posts aside from the four other people they'd gone through the training with. They'd get bored and quit blogging, just as the next trainees came online.

There was never a tipping point -- ''never a moment when two people who never knew each other could begin discussing something,'' as Clay Shirky, a professor at New York University who was hired to consult on the project, explained to me. For the intelligence agencies to benefit from ''social software,'' he said, they need to persuade thousands of employees to begin blogging and creating wikis all at once. And that requires a cultural sea change: persuading analysts, who for years have survived by holding their cards tightly to their chests, to begin openly showing their hands online.


IS IT POSSIBLE TO RECONCILE the needs of secrecy with such a radically open model for sharing? Certainly, there would be merit in a system that lets analysts quickly locate like-minded colleagues around the world to brainstorm new ideas about how the Iraqi insurgency will evolve. But the intelligence agencies also engage in covert operations that ferret out truly incendiary secrets: the locations of Iranian nuclear facilities, say, or the name of a Qaeda leader in Pakistan. Is this the sort of information that is safe to share widely in an online network?

Many in the intelligence agencies suspect not. Indeed, they often refuse to input sensitive intel into their own private, secure databases; they do not trust even their own colleagues, inside their own agencies, to keep their secrets safe. When the F.B.I. unveiled an automated case-support system in 1995, agents were supposed to begin entering all information from their continuing cases into it, so that other F.B.I. agents could benefit from the collected pool of tips. But many agents didn't. They worried that a hard-won source might be accidentally exposed by an F.B.I. agent halfway across the country. Worse, what would happen if a hacker or criminal found access to the system?

These are legitimate concerns. After the F.B.I. agent Robert Hanssen was arrested for selling the identities of undercover agents to Russia, it turned out he had found their names by trawling through records on the case-support system. As a result, many F.B.I. agents opted to keep their records on paper instead of trusting the database -- even, occasionally, storing files in shoeboxes shoved under their desks. ''When you have a source, you go to extraordinary lengths to protect their identities,'' I. C. Smith, a 25-year veteran of the bureau, told me. ''So agents never trusted the system, and rightly so.''

Worse, data errors that allow information to leak can often go undetected. Five years ago, Zalmai Azmi -- currently the chief information officer of the F.B.I. -- was working at the Department of Justice on a data-sharing project with an intelligence agency. He requested data that the agency was supposed to have scrubbed clean of all classified info. Yet when it arrived, it contained secret information. What had gone wrong? The agency had passed it through filters that removed any document marked ''secret'' -- but many documents were stamped ''SECRET,'' in uppercase, and the filter didn't catch the difference. The next time Azmi requested documents, he found yet more secret documents inadvertently leaked. This time it was because the documents had ''S E C R E T'' typed with a space between each letter, and the filter wasn't programmed to catch that either.

A spy blogosphere, even carefully secured against intruders, might be fundamentally incompatible with the goal of keeping secrets. And the converse is also true: blogs and wikis are unlikely to thrive in an environment where people are guarded about sharing information. Social software doesn't work if people aren't social.

Virtually all proponents of improved spy sharing are aware of this friction, and they have few answers. Meyerrose has already strained at boundaries that make other spies deeply uneasy. During the summer, he set up a completely open chat board on the Internet and invited anyone interested to participate in a two-week-long discussion of how to improve the spy agencies' policies for acquiring new technology.

The chat room was unencrypted and unsecured, so anyone could drop in and read the postings or mouth off. That way, Meyerrose figured, he'd be more likely to get drop-ins by engineers from small, scrappy start-up software firms who might have brilliant ideas but no other way to get an audience with intelligence chiefs. The chat room provoked howls of outrage. ''People were like, 'Hold it, can't the Chinese and North Koreans listen in?' '' Meyerrose told me. ''And, sure, they could. But we weren't going to be discussing state secrets. And the benefits of openness outweigh the risks.''

For something like Intellipedia, though, which trafficks in genuinely serious intelligence, hard decisions had to be made about what risks were acceptable. Fingar says that deeply sensitive intel would never be allowed onto Intellipedia -- particularly if it was operational information about a mission, like a planned raid on a terrorist compound. Indeed, Meyerrose's office is building three completely separate versions of Intellipedia for each of the three levels of secrecy: Top Secret, Secret and Unclassified. Each will be placed on a data network configured so that only people with the correct level of clearance can see them -- and these networks are tightly controlled, so sensitive information typed into the Top Secret Intellipedia cannot accidentally leak into the Unclassified one.

But will this make the Intellipedia less useful? There are a few million government employees who could look at the relatively unsecret Intellipedia. In contrast, only a few thousand intelligence officials qualify for a Top Secret clearance, and thus will be allowed into the elite version. This presents a secrecy paradox. The Unclassified Intellipedia will have the biggest readership and thus will grow the most rapidly; but if it's devoid of truly sensitive secrets, will it be of any use?

Fingar says yes, for an interesting reason: top-secret information is becoming less useful than it used to be. ''The intelligence business was initially, if not inherently, about secrets -- running risks and expending a lot of money to acquire secrets,'' he said, with the idea that ''if you limit how many people see it, it will be more secure, and you will be able to get more of it. But that's now appropriate for a small and shrinking percentage of information.'' The time is past for analysts to act like ''monastic scholars in a cave someplace,'' he added, laboring for weeks or months in isolation to produce a report.

Fingar says that more value can be generated by analysts sharing bits of ''open source'' information -- the nonclassified material in the broad world, like foreign newspapers, newsletters and blogs. It used to be that on-the-ground spies were the only ones who knew what was going on in a foreign country. But now the average citizen sitting in her living room can peer into the debates, news and lives of people in Iran. ''If you want to know what the terrorists' long-term plans are, the best thing is to read their propaganda -- the stuff out there on the Internet,'' the W.M.D. analyst told me. ''I mean, it's not secret. They're telling us.''

Fingar and Andrus and other intelligence thinkers do not play down the importance of covert ops or high-tech satellite surveillance in intercepting specific jihadist plots. But in a world that is awash in information, it is possible, they say, that the meaning of intelligence is shifting. Beat cops in Indiana might be as likely to uncover evidence of a terror plot as undercover C.I.A. agents in Pakistan. Fiery sermons printed on pamphlets in the U.K. might be the most valuable tool in figuring out who's raising money for a possible future London bombing. The most valuable spy system is one that can quickly assemble disparate pieces that are already lying around -- information gathered by doctors, aid workers, police officers or security guards at corporations.

The premise of spy-blogging is that a million connected amateurs will always be smarter than a few experts collected in an elite star chamber; that Wikipedia will always move more quickly than the Encyclopaedia Britannica; that the country's thousand-odd political bloggers will always spot news trends more quickly than slow-moving journalists in the mainstream media. Yet one of the most successful new terrorism-busting spy organizations since 9/11 does in fact function like a star chamber. The National Counterterrorism Center was established by Congress in 2004 and charged with spotting the most important terrorism threats as they emerge. The counterterrorism center is made up of representatives from every intelligence agency -- C.I.A., F.B.I., N.S.A. and others -- who work together under one roof. Each analyst has access to details particular to his or her agency, and they simply share information face to face. The analysts check their personal networks for the most dire daily threats and bring them to the group. In three meetings a day, the officials assess all theintel that has risen to their attention -- and they jointly decide what the nation's most serious threats are. ''We call it carbon-based integration,'' said William Spalding, the center's chief information officer.

When I raised the idea of collaborative tools like blogs and wikis, Spalding and Russ Travers, one of the center's deputy directors, were skeptical. The whole reason the center works, they said, is that experts have a top-down view that is essential to picking the important information out of the surrounding chatter. The grass roots, they've found, are good at collecting threats but not necessarily at analyzing them. If a lot of low-level analysts are pointing to the same inaccurate posting, that doesn't make it any less wrong.''

The key is to have very smart people culling'' the daily tips, Travers told me. In October, for example, nervous rumors that a football stadium in the United States would be subject to a nuclear attack flooded the National Counterterrorism Center; analysts there immediately suspected it was spurious. ''The terrorist problem has the worst signal-to-noise ratio,'' Travers said. Without the knowledge that comes from long experience, he added, a fledgling analyst or spy cannot know what is important or not. The counterterrorism center, he said, should decide which threats warrant attention. ''That's our job,'' he said.

The Spying 2.0 vision has thus created a curious culture battle in intelligence circles. Many of the officials at the very top, like Fingar, Meyerrose and their colleagues at the office of the director of national intelligence, are intrigued by the potential of a freewheeling, smart-mobbing intelligence community. The newest, youngest analysts are in favor of it, too. The resistance comes from the ''iron majors'' -- career officers who occupy the enormous middle bureaucracy of the spy agencies. They might find the idea of an empowered grass roots to be foolhardy; they might also worry that it threatens their turf.

And the critics might turn out to be right. As Clay Shirky of N.Y.U. points out, most wikis and blogs flop. A wiki might never reach a critical mass of contributors and remain anemic until eventually everyone drifts away; many bloggers never attract any attention and, discouraged, eventually stop posting. Wikipedia passed the critical-mass plateau a year ago, but it is a rarity. ''The normal case for social software is failure,'' Shirky said. And because Intellipedia is now a high-profile experiment with many skeptics, its failure could permanently doom these sorts of collaborative spy endeavors.

There is also the practical question of running a huge civil-service agency where you have to assess the performance of your staff. It might be difficult to measure contributions to a wiki; if a brilliant piece of analysis emerges from the mob, who gets credit for it? ''A C.I.A. officer's career is advanced by producing reports,'' notes David Weinberger, a fellow at the Harvard Berkman Center for the Internet and Society, who consulted briefly with the C.I.A. on its social software. ''His ability is judged by those reports. And that gets in the way of developing knowledge socially, where it becomes very difficult to know who added or revised what.''

In addition, civil libertarians are alarmed by the idea of spies casually passing sensitive information around from one agency to another. ''I don't want the N.S.A. passing on information about innocent Americans to local cops in San Diego,'' Weinberger said. ''Those laws exist for good reasons.''

In many ways, the new generation of Web-savvy spies frames the same troubling questions as the Patriot Act, which sought to break down the barriers preventing military spy agencies from conducting operations inside the United States, on American citizens, and then sharing that information with domestic groups. On a sheerly practical level, it makes sense to get rid of all barriers: why not let the N.S.A. wiretap American conversations? Vice President Cheney has argued forcefully that these historical barriers between agencies hobble the American military and intelligence forces; the Patriot Act was designed in part to eliminate them. Terrorist groups like Al Qaeda heed no such boundaries, which is precisely why they can move so quickly and nimbly.

Then again, there's a limit to how much the United States ought to emulate Al Qaeda's modus operandi. ''The problems the spies face are serious; I sympathize with that,'' Shirky told me. ''But they shouldn't be wiping up every bit of information about every American citizen.'' The Pentagon's infamous Total Information Awareness program, which came to light in 2002, was intended to scoop up information on citizens from a variety of sources -- commercial purchase databases, government records -- and mine it for suggestive terrorism connections. But to many Americans, this sort of dot-connecting activity seemed like an outrageous violation of privacy, and soon after it was exposed, the program was killed. James X. Dempsey, director of the Center for Democracy and Technology, maintains that the laws on spying and privacy need new clarity. The historic morass of legislation, including the Patriot Act, has become too confusing, he says; both spies and the public are unsure what walls exist. While Dempsey agrees that agencies should probably be allowed to swap more information than they currently do, he says that revamped rules must also respect privacy -- ''otherwise, we'll keep on producing programs that violate people's sense of what's right, and they'll keep getting shut down.''

For all the complaints about hardware, the challenges are only in part about technology. They are also about political will and institutional culture -- and whether the spy agencies can be persuaded to change. Some former intelligence officials have expressed skepticism about whether Meyerrose and Fingar and their national-intelligence colleagues have the clout and power to persuade the agencies to adopt this new paradigm. Though D.N.I. officials say they have direct procurement authority over technology for all the agencies, there's no evidence yet that Meyerrose will be able to make a serious impact on the eight spy agencies in the Department of Defense, which has its own annual $38 billion intelligence budget -- the lion's share of all the money the government spends on spying. When I spoke to Wilson P. Dizard III, a writer with Government Computer News who has covered federal technology issues for two decades, he said, ''You have all these little barons at N.S.A. and C.I.A. and whatever, and a lot of people think they're not going to do what the D.N.I. says, if push comes to shove.''

Today's spies exist in an age of constant information exchange, in which everyday citizens swap news, dial up satellite pictures of their houses and collaborate on distant Web sites with strangers. As John Arquilla told me, if the spies do not join the rest of the world, they risk growing to resemble the rigid, unchanging bureaucracy that they once confronted during the cold war. ''Fifteen years ago we were fighting the Soviet Union,'' he said. ''Who knew it would be replicated today in the intelligence community?''

Posted by Clive Thompson at 01:42 PM
Comments working now!

I fixed the comments. It turns out the permissions on my commenting script got b0rked -- not sure why, but they're better now.

Thanks to Anthony, Ronnie, Adam, Jesse, Jemaleddin, Zac, and others for helping point me to the fix!

Posted by Clive Thompson at 01:38 PM
December 03, 2006
Comments busted on my blog. Anyone know how to fix this?

Sheesh. I finally start blogging after two months, and for some reason the comments seem to be busted! Several folks have emailed me to say they've tried to post and can't.

Whenever anyone tries to post, they get this error message:

Forbidden
You don't have permission to access /mt3/mt-comments.cgi on this server.

Anyone know what might have gone wrong here? If you've had this error on your instal of Movable Type and think you know how I can fix it, please email me for my eternal thanks!

Posted by Clive Thompson at 07:39 PM
MeTube!











This is pretty excellent: Apparently, one of my magazine articles was part of the inspiration for YouTube!

Recently, Jawed Karim, one of the three cofounders of the site, gave a speech at the ACM conference at the University of Illinois at Urbana-Champaign. He talked about the many trends that he was tracking in 2004 that led up to the "aha" moment where he envisioned YouTube. One of these moments was when Karim read "The Bittorrent Effect," a story I wrote for Wired magazine.

In the piece, I described the infamous episode of Crossfire in which Jon Stewart showed up and reamed out the two hosts for "hurting America" with their formulaic, gormless Punch-and-Judy approach to modern political debate. The clip of Stewart's rant was quickly ripped, posted online, and passed around with such speed that -- as best as I could calculate -- over 2.5 million people saw it online. Then, as I wrote:

By contrast, CNN's audience for Crossfire was only 867,000. Three times as many people saw Stewart's appearance online as on CNN itself.

If enough people start getting their TV online, it will drastically change the nature of the medium. Normally, the buzz for a show builds gradually; it takes a few weeks or even a whole season for a loyal viewershi