Suppose you’ve been good (or sort of good anyway), and you have a huge stack of CD-ROMs (or DVDs) with backups and archives of your old files. Great. But how can you find anything? I solved this problem today by making an index of all the files stored on these disks using a few simple GNU command line tools.

I sometimes think that search tools are like my local bus: none comes along for ages and then three turn up in quick succession. For quite some time Beagle and Kat have been meeting the needs of users like you and me who fill up their hard drives with the results of our internet meanderings and because we have been remiss in keeping those drives well organized we eventually have to use search tools to find that PDF or HTML article we spent an eternity looking for.

Every once in awhile, I just get a hare-brained notion. Today’s was, why do we use a central website for doing internet searches at all? Why Google?

Consider the success of the Planetary Society’s distributed SETI project, and the distributed computing architecture that resulted from it. Consider the success of swarming download technology like BitTorrent. Consider how simple a basic web spider could be. Consider the efficiency of spidering networks locally. Consider the architecture of DNS.

See a pattern?