Dig this: A group of Hewlett Packard researchers studied 200,000 emails sent on their company’s internal network. They analyzed them to see how they clustered, and discovered that it’s pretty easy to figure out what little subgroups exist within a larger organization. As a story in the New Scientist puts it:

To pick them out, the researchers used a computer algorithm that looks for the critical links that form bridges between separate groups - what the team calls links with high “betweenness”. By severing these links one by one, the algorithm gradually isolates people into different communities of groups who are emailing each other. …

The technique revealed 66 communities at the lab. And when the researchers compared the community members with the company organisation charts, they found that 49 of them contained people who all worked in the same department. In most of the others, the people were collaborating on a project.

Simple enough — when you look inside an organization, you’ll find a bunch of groups. Where it gets interesting is when you look outside, into the full Internet. Would it be possible to examine email movement to figure out who “knows” who? Or, more importantly, who’s the ringleader of a group?

In a second investigation, the team plotted the same network of emails using a standard algorithm that, in effect, tries to arrange it in the least tangled way possible. This showed that the managers, including the director, tended to cluster in the middle. “This approach puts in the middle the people who have the most diverse range of contacts in the organisation - and these tend to be the leaders,” says Tyler.

An interesting way to hunt for the leader of a terrorist cell, to be sure. But also a superb way to run a dragnet on ever more totally innocent people, which is the strange direction our security apparatuses seem to be going these days. I can only imagine how much the Carnivore guys are slavering over these sorts of techniques. The HP guys themselves seem fairly agnostic as to how one could use this technique; they’re just scientists, and this is good science — you can download a PDF of their paper about the experiment here.

(Thanks to Coder Log for pointing this one out!)