Last June, Ed Felten and his team at Princeton published Government Data and the Invisible Hand. They argued that government agencies should stop trying to present information to the public, because they aren't good at it. Instead, they should just publish the data and let others present it.
A few weeks later, the National Counterterrorism Center published a huge data set on its Web site. The Worldwide Incidents Tracking System has a few thousand records of violent acts going back to 2004.
The entire data set is downloadable as a 6MB XML file. This is pretty cool. If you're a coder and you know Processing, ActionScript, or some other data visualization language, I'm begging you to take this data set and do something cool with it. I have two reasons:
- First, the NCTC probably has very little resources when it comes to data visualization. This huge data set could reveal some unknown patterns about terrorism, if only it were presented in the right way. The XML file includes information on location, the background of the assailant, the type of target, weapons used, the number of casualties, and the cost of the damage. Lots could be learned about new trends in violent extremism: are some geographic areas cooling off, while others are heating up? Is a particular locale shifting from one tactic to another? Does a certain group have a new preferred weapon? Even if you don't support the idea of a "war on terror" or approve of the way it's been carried out, such knowledge benefits everyone interested in peace.
- Second, and more importantly, we need to show government agencies that posting such data is worth their while. This is a pretty radical experiment for a national security agency; it's hard enough just getting simple data on routine legislation. Now that a secretive agency has taken such a leap, we need to show them what they stand to gain from such experiments. We all want government agencies to be more open with their data. They'll comply much more quickly if we show them there's a fast, tangible benefit to doing so.
Comments
more of a flex project than a flash one...
Besides visualizing the data, it would be a matter of creating more revealing searches than the completely open-parameter searches currently on the gov web site. For instance you are suggesting searches that identify trends. That could take some analytic chops.