24 hours

Seb Paquet writes Seb's Open Research, a blog with "pointers and thoughts on the evolution of knowledge sharing and scholarly communication." He wrote me from New Brunswick that he planned to be in San Francisco today and we decided to rendezvous.

Late Wednesday Seb sent me an email that he was going to a party in my neighborhood in Berkeley the next evening and perhaps we could meet there. So I put on my Santa suit, wandered over to Jerry Michalski's house (I thought he lived in Sausalito; he's actually a neighbor), and met an absolutely wonderful group of people, some of whom I knew through their blogs.

The story continues...

At the party, Gordon Moore (no, not that Gordon Moore), invited us to drop by the Internet Archive for lunch the next day. Seb was new to San Francisco, so this morning I led him on a whirlwind tour on the way to lunch. Among other things, we tangled with Union Square, the Post St. shops, Chinatown, North Beach, cable cars, the Marina, the Palace of Fine Arts, the Presidio, the Palace of the Legion of Honor, Seal Rock, Great Beach, and Golden Gate Park.

The Internet Archive

It's a great contrast: a century-old, clapboard Army building housing an altruistic, hyper-high tech operation. Inside, Brewster Kahle and a dozen helpers create, maintain, and move forward the Internet Archive.

Brewster aims to capture and preserve all the books, magazines, television, the web, software, and music created by humankind, and to make it accessible to the entire world. He thinks of it as "making the free world work." It's a 25-year goal.

In addition to the Archive staff, clustered around the luncheon table were a chap from the National Library of Iceland, another from the National Library of Norway, two people from the Project Gutenberg Distributed Proofreaders, a guy from SBC Global, a volunteer from Boston who is helping the Archive capture music, and others.

The Archive is moving its computers into a new data center. The fellows moving the PCs joked about carting around "100 Terabytes in a U-Haul." The Archive runs on a complex of nearly a thousand computers. Their typical computer includes four 250 gig hard drives, a terabyte in all, and costs about $1,400. They consume about $500 worth of electricity every month.

From the Archive's site:

Why the Archive is Building an 'Internet Library'

Libraries exist to preserve society’s cultural artifacts and to provide access to them. If libraries are to continue to foster education and scholarship in this era of digital technology, it’s essential for them to extend those functions into the digital world.

Many early movies were recycled to recover the silver in the film. The Library of Alexandria — an ancient center of learning containing a copy of every book in the world — was eventually burned to the ground. Even now, at the turn of the 21st century, no comprehensive archives of television or radio programs exist.

But without cultural artifacts, civilization has no memory and no mechanism to learn from its successes and failures. And paradoxically, with the explosion of the Internet, we live in what Danny Hillis has referred to as our "digital dark age."

The Internet Archive is working to prevent the Internet — a new medium with major historical significance — and other "born-digital" materials from disappearing into the past. Collaborating with institutions including the Library of Congress and the Smithsonian, we are working to preserve a record for generations to come.

Stewart Brand has written:

"The Internet Archive is a service so essential that its founding is bound to be looked back on with the fondness and respect that people now have for the public libraries seeded by Andrew Carnegie a century ago.... Digitized information, especially on the Internet, has such rapid turnover these days that total loss is the norm. Civilization is developing severe amnesia as a result; indeed it may have become too amnesiac already to notice the problem properly. The Internet Archive is the beginning of a cure — the beginning of complete, detailed, accessible, searchable memory for society, and not just scholars this time, but everyone."

One amazing aspect of the Internet Archive is its reliance on volunteers. The fellow assembling the music archive does it as a labor of love. Today was the first time he had met Brewster or visited the Archive. Similarly, Project Gutenberg's Distributed Proofreaders spreads the task of proofreading amongst five hundred active volunteers. Some people check a page a day, others complete dozens, and some folks do this almost fullltime. Interested? They'd be glad to have you join in "preserving history one day at a time." Thus far, PGDP has proofed a million pages. They've posted 10,000 public domain books to Project Gutenberg. Charles Franks says they're tracking their target of proofing a million books in ten years. The strength of numbers at work--along with the genius of chopping the work up into small pieces....

Conversations with Jerry Michalski, Jerry's mom, Steve Larsen (Net Perceptions), Peter Merholz, Seb Paquet, the Archive people, and dozens of others have generated so many ideas and connections that my head feels about ready to pop. I'm going to have my morning coffee, browse through the New York Times, and let "the boys in the back room" process my neural firings.

Well, I find I have to get out a few thoughts.

I am filled with optimism that we can make the world a better place. Folks like Jerry and Brewster are going to help us do it.

Everything in the world is connected or becoming so. At present it's like the "Internet cloud." You don't see the lines of connection but you trust that they are there. I'm beginning to perceive something parallel, sort of "Reality Soup." I appreciate that everything (systems, people, places) is connected, I don't see most of the connections, but just realizing I'm in the soup simplifies my worldview.

Now, I'm going to go get that coffee.

Posted by Jay Cross at December 11, 2003 10:43 PM | TrackBack