Blog Pile

Web X.X and Our History of Collecting URLs

Since not long after the day in October 1993 when a colleague handed my a floppy disk (remember those?) labeled “Mosaic”, our office has had a continuously running web server. We have gone through several iterations of trying to build collections of web resources for educators, and in going through some old files, I got a bit nostalgic. Just a bit.

Our latest iteration might be construed as “Web 2.0”, but let’s roll back the decimals…

Web 0.9: Old School Hand Spun HTML (1994-1995):

Mcli-Urls-Two-Oh4

World Wide Web News was a the old simple HTML with a few graphics that we published on our site and then advertised by email. The issue above dated September 14, 1994 covers web browsers, our web server running on a Mac SE/30, and some web links to Frog dissection, AskEric Virtual Librarian, The Pompeii Forum Project, etc (links are most likely long gone).

In later 1995, we went wild and crazy and used those background tile images, the cutting edge NetScape table tags (look at June 1, 1995 for “New: Jazzed Look for NetScape 1.1”.

Mcli-Urls-Two-Oh5

And we picked a few “Top of the Web” picks. The web seemed pretty tiny then, looking back.

In late 1995, we began trying to cull these into a subject organized structure of “Hot Links to the World Wide Web”, where we tried like the first Yahoos, to put links into a navigation tree. Again it was all hand coded HTML.

Mcli-Urls-Two-Oh3
Mcli-Urls-Two-Oh2

We also had a set of suggested web starting points for general information.

Web 1.0: Indexed Search And The Second Crudest Database (1996-1997):
Recognizing some of the folly of trying to manually get links into categories, we moved to different system- using a web server indexing code (GLIMPSE?) and some perl code, to have all the data stored in a text file database that could be searched, and make the information displayed dynamic. With the help of my first part-time student programmer (thanks Kurt) we created the 451F collection (with ap0logies to Ray Bradbury):

Mcli-Urls-Two-Oh1

What design! Using those new fangled frame displays, we could keep a search/navigation area on top. All entries were more or less “tagged” with the categories in a drop down menu, and key words could be searched against the whole collection, or a category.

At this point, though, I was the only one that could add links to the collection. We started to see some early issues of links going bad.

Web 1.1: Bag of URLs with Submission and Content “Management” (1997-2005):
We began to expand our methods of creating web resource sites, to include web forms for people to add new sites to our collection. In the beginning, we naively had an open submission, but quickly learned we needed our own admin tools to preview the submissions, edit the old ones, etc. The “Bag of URLs” was set up so we could collect 20-40 new sites, descriptions in a holding bin, then when ready, we had a web interface for publishing them as a new web page, re-creating the main page with the current content, and generating an email message that could go out announcing the new issue (we used a listserv so interested persons could self subscribe). Here is an example from May 2000:

Mcli-Urls-Two-Oh6

This ran for quite some time, though in the last few years, I got less frequent with compiling new issues. Between 1996 and 2005, we added 4721 sites, and all of the sites added could be keyword searched. Somewhere in the early 2000 years, I created a bookmarklet tool to make added them pretty simple from my web browser. We continued to have troubles with links going bad, and on perhaps a yearly basis, took the time to run all the links through a checker and tried to remove the dead ones. On more than one occasion, we were notified that links that once were valid content had been purchased by porn outfits.

We were still basically running this from a large tab-delimited text file that was indexed by a unix search tool.

About a year ago, the volume of legitimate sites coming in to spammers trying to dump links in our form got to be about 80:1, so I cut off the open form submission, adding a required password. It also became clear that even the manual effort of dumping links into a form, was taking too much time.

Web 2.0: Bag of URLs Fueled By Del.icio.us (2006-):
Like a thunderclap, it struck me that it was much more reasonable to maintain this site simply by tagging sites I was already tagging for my own collection with a special tag for items I wanted to display on this site. So the current version merely uses the JavaScript dynamic display code from del.icio.us to show everything I have tagged with “urlbag”:

Mcli-Urls-Two-Oh7

At this time, it is only picking up my own tags; I’ve yet to sort out the benefit of public tagging for what is a rather specialized collection. There are issues of tag pollution… It might be nice if there was some way to say aggregate the del.icio.us tags of say, a group of users I trust, rather than all the crowd on the site.

And at this point, I never have to edit the main site again. I like that.

What a long, link loaded road it has been.

If this kind of stuff has value, please support me by tossing a one time PayPal kibble or monthly on Patreon
Become a patron at Patreon!
Profile Picture for CogDog The Blog
An early 90s builder of web stuff and blogging Alan Levine barks at CogDogBlog.com on web storytelling (#ds106 #4life), photography, bending WordPress, and serendipity in the infinite internet river. He thinks it's weird to write about himself in the third person. And he is 100% into the Fediverse (or tells himself so) Tooting as @cogdog@cosocial.ca

Comments

  1. Having just found my floppy disk copy of World Wide Web Weaver, I love the history as I could relate to it totally BUT… “I never have to edit the main site again”?? What are you thinking? The whole article is about how you were always on the cutting edge and suddenly you think that one small technology is going to be the be all and end all of page creation? We wish :-). Is that Web 3.0 on the horizon or am I just getting older?

  2. I was unclear- I was referring to one less web page area I have to edit (among the 100+ other different sites on our server. I had about 3 or four other onece very active web resource sites that had a similar evolution. The real lesson is that the collection of “web resources” by yet another list of web sites, is well, silly.

    And nor am I really always rtight about technology.

    PS- if you are nostalgic for old web browsers, see http://browsers.evolt.org — fire up a copy of Mosais 0.93 and take a click down memory lane.

Comments are closed.