Since not long after the day in October 1993 when a colleague handed my a floppy disk (remember those?) labeled “Mosaic”, our office has had a continuously running web server. We have gone through several iterations of trying to build collections of web resources for educators, and in going through some old files, I got a bit nostalgic. Just a bit.
Our latest iteration might be construed as “Web 2.0”, but let’s roll back the decimals…
Web 0.9: Old School Hand Spun HTML (1994-1995):
World Wide Web News was a the old simple HTML with a few graphics that we published on our site and then advertised by email. The issue above dated September 14, 1994 covers web browsers, our web server running on a Mac SE/30, and some web links to Frog dissection, AskEric Virtual Librarian, The Pompeii Forum Project, etc (links are most likely long gone).
In later 1995, we went wild and crazy and used those background tile images, the cutting edge NetScape table tags (look at June 1, 1995 for “New: Jazzed Look for NetScape 1.1”.
And we picked a few “Top of the Web” picks. The web seemed pretty tiny then, looking back.
In late 1995, we began trying to cull these into a subject organized structure of “Hot Links to the World Wide Web”, where we tried like the first Yahoos, to put links into a navigation tree. Again it was all hand coded HTML.
We also had a set of suggested web starting points for general information.
Web 1.0: Indexed Search And The Second Crudest Database (1996-1997):
Recognizing some of the folly of trying to manually get links into categories, we moved to different system- using a web server indexing code (GLIMPSE?) and some perl code, to have all the data stored in a text file database that could be searched, and make the information displayed dynamic. With the help of my first part-time student programmer (thanks Kurt) we created the 451F collection (with ap0logies to Ray Bradbury):
What design! Using those new fangled frame displays, we could keep a search/navigation area on top. All entries were more or less “tagged” with the categories in a drop down menu, and key words could be searched against the whole collection, or a category.
At this point, though, I was the only one that could add links to the collection. We started to see some early issues of links going bad.
Web 1.1: Bag of URLs with Submission and Content “Management” (1997-2005):
We began to expand our methods of creating web resource sites, to include web forms for people to add new sites to our collection. In the beginning, we naively had an open submission, but quickly learned we needed our own admin tools to preview the submissions, edit the old ones, etc. The “Bag of URLs” was set up so we could collect 20-40 new sites, descriptions in a holding bin, then when ready, we had a web interface for publishing them as a new web page, re-creating the main page with the current content, and generating an email message that could go out announcing the new issue (we used a listserv so interested persons could self subscribe). Here is an example from May 2000:
This ran for quite some time, though in the last few years, I got less frequent with compiling new issues. Between 1996 and 2005, we added 4721 sites, and all of the sites added could be keyword searched. Somewhere in the early 2000 years, I created a bookmarklet tool to make added them pretty simple from my web browser. We continued to have troubles with links going bad, and on perhaps a yearly basis, took the time to run all the links through a checker and tried to remove the dead ones. On more than one occasion, we were notified that links that once were valid content had been purchased by porn outfits.
We were still basically running this from a large tab-delimited text file that was indexed by a unix search tool.
About a year ago, the volume of legitimate sites coming in to spammers trying to dump links in our form got to be about 80:1, so I cut off the open form submission, adding a required password. It also became clear that even the manual effort of dumping links into a form, was taking too much time.
Web 2.0: Bag of URLs Fueled By Del.icio.us (2006-):
At this time, it is only picking up my own tags; I’ve yet to sort out the benefit of public tagging for what is a rather specialized collection. There are issues of tag pollution… It might be nice if there was some way to say aggregate the del.icio.us tags of say, a group of users I trust, rather than all the crowd on the site.
And at this point, I never have to edit the main site again. I like that.
What a long, link loaded road it has been.
The post "Web X.X and Our History of Collecting URLs" was originally thawed from a previous ice age and melted at CogDogBlog (http://cogdogblog.com/2006/03/web-urls/) on March 3, 2006.