I have great news to share- given the generosity of people who have donated financial support, and one anonymous donor in particular, I have sufficient funds to keep Feed2JS running at least through June 2013, and maybe longer.

For those not following this untrending topic, Feed2JS is a service I developed way back in 2003 while at the Maricopa Community Colleges- it exists as a platform for people to set up cut and past code to create a dynamic display in their own web sites of content generated by RSS.

As a matter of historical note- the idea for this was inspired by a site David Carter-Tod had created- the Wytheville Community College News Center. It was that example where I saw how via JavaScript a server script could could be called, passed a URL for an RSS feed, and in turn return the content of the feed via JS output statements.

David’s site used ASP; I decided to try my hand at my own with PHP and an old RSS parser called Onyx- the first site called rss2js was announced May 7, 2003. I later changed the name to Feed2JS which ran a number of years on a server at Maricopa, and then was generously offered free hosting by an ISP called Modevia.

Fast forward to September 27 of this year- While chilling on my sister’s boat in the Chesapeake Bay, IO got a few email questions asking what was wrong with the site. I checked, and indeed it was kaput. Bottom line, I discovered the company that bought Modevia had shut off the server (after getting in touch with them, they turned it on a few days to give me time to set up a new host).

I put out a red alert as I could not see being able to pay for a dedicated server. Feed2JS generates so much traffic, your garden variety $20 a month shared web server would be squashed by the demand – it required a dedicated server that runs $180 a month.

I put up some donation buttons, and added a little annoying link in the output content.

The donations came in, and a steady stream of them through October. Most recently, I was exploring the option of putting text add links on the main site.

And then a freedom moment happened.

cc licensed ( BY NC SD ) flickr photo shared by Rosh PR

I was contacted by someone who wishes to remain anonymous, that is offering enough extra funds to keep Feed2JS going a long time. This person wants no credit, beyond me writing this post.

What this means is that the main Feed2JS is secure for at least 18 months, maybe longer. I also removed the “supported by Feed2JS” link that was in the output the last few weeks. There are no ads at all on the site.

It means the service will keep going strong and the server costs is not a problem for the mid-term future.

This is great news, eh?

What I still seek, are some people to step in and help with some of the coding and working on improving the site. I set up a Google doc to collect some interests- and have not seen much action. I probably won’t be doing much beyond trying to address bugs reports that come in.

So who wants to lend some PHP code chops to this project? Bueller? Bueller?

PS- I owe David Carter-Tod a huge apology- I was in Blacksburg VA this October and completely biffed on remembering to contact him. I shall return and make amends in 2012

I have gotten an impressive stream of support, donations with my request to help support Feed2JS, enough to keep the site up and running (which apparently has been stable). But I’ve not been comfortable depending on a future path of donation support, although y’all have demonstrated the potential here.

It ought to be free.

I had a few nibbles of interest of offers to support the costs (in exchange for some visible sponsorship on the main site), with one that looked very viable, but still up in the air.

But a new one is on the table, and it would pay for the ongoing server costs in exchange for text links ads in the footer of the Feed2JS site. I felt a bit squeamish, but it seems pretty minimal; I would get to know the site listed since they are static text. And frankly it seems like no impact for the people who use the site. I would still provide only a “powered by feed2js” in the output, and even that can ve easily hidden by CSS.

I know some may object, who can cast a stone my way for sucking up ad paid route. But to me, it takes away the paying for the server as an issue, and can make room to focus on the ongoing effort to get people lined up to work on improving the site.

Which, ahem, has gotten no responses in an open google doc I set up. Maybe that was not the best route to solicit input. I guess I need to learn some git and set up a hub.

So I am asking the question of anyone who cares, if a few text ads at the bottom of feed2js.org is the incarnation of the devil. I’ve pretty much made up my mind, but wanted at least to put out the question.

This has been beyond my expectations as a response to my call for help to keep Feed2JS going. I actually had never looked much to see where it was being used (D’Arcy would claim its mostly spam), since the site does not track anything (working on a log to count the number of feed cached each day, estimated at 15,000) but Google reports 190,000 results for a search string that indicate usage from the main server alone.

Then there were the messages people shared when donating to help with the server costs. I’ve been collecting the advice, suggestions, offers etc, and after a really helpful discussion yesterday with Scott Leslie, can outline a few plans for the future. They are ones in which I am asking for some help, especially in advancing the code development.

But first as an update. Aaron Axelsen again has been my server hero. The rackspace could server we set up first has not keeping up well with the demand, and the usage costs were increasing. He suggested going for a dedicated server at softlayer, and migrated it quickly for me, and it seems to be much more responsive and lest prone to crashes. And the cost is lower, fixed at $180/month.

While I first expressed an interest in handing over the project to some other party, I have decided this is my baby, and I would like a role in seeing it go forward. But I lack the code skills to take it much farther, and would like to see some others step in who can carry out this end of it.

That said, the response so far has been incredible, with over $1000 raised, meaning enough for now to go about 5 months (including paying for the first month at rackspace). Some options going forward include:

• Keeping it going on donation basis. At least one person has promosed to pitch in every month.
• Getting a major sponsor to support it. I’m in conversation with one company that has offered this as a possible solution, but it is not confirmed. Most like this would mean some prominent banner or credit on the main site (not in the feeds) however there would now always likely be a link on the output like “powered by Feed2JS” with a link back here.
• Some sort of subscription service, e.g. for a small fee, you could have that credit link removed. Or some other perk.
• I have also added a flattr widget to the site for people who play there– see http://flattr.com/thing/411163/feed2js

But what I really need the assistance with is some people to volunteer to help in the code development/improvement. I’ve gone about as far as my PHP skills will carry me and there are a bunch of things the site could use. It could also stand a design refresh, so I am eager to find some help there.

Ideally, the main site would be the hub, but I’d like to have a list of reliable mirror sites that we could refer people to go for access to the same service.

I’ve outlined a few suggestion areas in a Google Doc I set up to collect ideas as well as contacts for people who’d like to be part of a crew to keep Feed2JS going.

I’m really excited that this little thing I started as an experiment in 2003 has gotten such wide use, this might be the biggest thing I’ve worked on. And I want to see it keep going strong.

Won’t you help? PLease?

cc licensed ( BY NC ) flickr photo shared by leeno

In doing this, a lot of traffic is routed through the Feed2JS server. More than I may have even guessed. I developed this and ran it from 2003 to maybe 2005 on a server at the Maricopa Community Colleges, and then had been hosted for free at Modevia Web Services after a kind offer from Aaron Axelsen. I never even had to look at the web server for the last 6 years.

In September, I got an email from apparently the company that bought out Modevia (I had been alerted this was happening) letting me know that they would be shutting down the server. CUt off. They did provide me a few days leeway to set up a new server.

Now I jnow next to nothing about setting up web servers, but with some advice from The Twitter, set up a cloud based server at Rackspace. I got it going, but it was apparently not the most efficient set up, so Aaron kindly configured a new server.

The thing is it runs a lot of traffic, and I am paying by the Gb. In two weeks, it’s been about over 600 Gb of inbound and outbound traffic. The load crashed the server in week 1, and I had to double the RAM.

If it is any measure of usage, the cache directory is cleared daily, and registeres at least 1500 different feeds a day it is serving, meaning it is used on at least that many web sites.

At this rate, its going to cost me at least $200 just for hosting in September (for those who will chime in about their $15 ISP, keep in mind Feed2JS would choke in a shared server environment, it needs its own server).

I can’t afford to do this going forward, and unless something happens in the next few weeks, I am going to have to pull the plug on it end of October. I have set up a paypal donation form, and to date have collected about $70.

But it really needs a major sponsor or some organization to take it over. I’m not too invested in running a web service. But I hate to leave people hanging- when the light’s go out on the server, people’s web pages will spin as it tries to load a script that is not there.

It really is not the optimum set up that there is a single server; a set of distributed ones would make much more sense.

So this is what’s happening. I have placed a notice atop the Feed2JS site:

This leads to a web page I made to outline what I have written here, along when three possible futures (see http://feed2js.org/index.php?s=support).

• Some generous benefactor is willing to cover the hosting costs. I am unwilling to inject advertising into the generated content, but there is no reason why this site could not be overt in recognizing this generosity.
• Some organization is willing to take over the management. This would mean hosting and also developing a plan to perhaps distribute the load across multiple hosts, as well as to party on efforts to develop the site. I’d even be willing to give up all ownership if Feed2JS is continued as a free service. Make me an offer.
• The free service goes away. I hate to see it happen, but this is a likely possibility. This would mean needing to find alternatives or look into installing your own local version- if you self host your web site, this is not that hard to do. See the Google Code site for all you need. I recommend this route anyhow as you will get much better performance from your own server.

Also, I was a bit reluctant to do this because it suggests a possibility of what some bad person could do with a server, in that I have modified the central script to append a small “please support Feed2JS” at the bottom of all calls to the script (yeah it reeks of advertising):

If you follow the link, it gives some info on how you can hide the nag message. It’s not really that tricky.

So why would you donate money if this might go away? I cant answer that either. I doubt a solution is running this on donations. But it would help me out in this lats stretch because IT”S COMING OUT OF MY POCKET and I DO NOT HAVE A JOB!

Better yet, if anyone has connections or ideas for some generous entity that would like to underwrite this or take it of my hands- please contact me ASAP.

I hate to kill Feed2JS, and frankly the code would remain available and open for anyone else to pick up and use/modify.

Please, help me find someone to adopt my little project.

Thanks.

Update: Sept 28 I am happy to report some generous donations came in last night, with a notable shout out to Anil Dash for his personal email. I am about 3/4 the way to covering costs for the next 2 months and have a nibble of interest from 2 sources.

I’m looking for someone, some group to take over the development for Feed2JS; I’ve taken it as far as I can possibly can.

It gets used a lot, more than I think I want to know. The server empties its cache nightly, but I just checked, and since the last nightly dumped, it has cached feeds from 38746 RSS sources. If you google the right bit of code (that part that is inserted into web pages to make it work, I see 50,000 some finds:

Yikes.

I have to give a huge shout out and thanks to Modevia Web Services, who have graciously provided free hosting for Feed2JS since early 2006, when luckily I moved it off of an aging Xserve at Maricopa Community Colleges (where it would have died after I left Maricopa in April 2006 and the old jade server went dark).

This road goes back more than 7 years, I can find myself blogging about it in April 2003 where I “munged up some code” called rss2js.

In my early years of sniffing around RSS I had come across some scripts for displaying news feeds embedded into a page that a guy named David Carter-Todd was running on a server at one of the Virginia Community Colleges, and a light bulb went off. His code was a server script that was passed an RSS feed URL, and it ran some code that returned the content as Javascript write statements, which means anyone could get RSS content to be seen in a web page.

I came across a PHP library called Onyx RSS (created by a guy named Ed Swindelles, the site is long gone http://www.readinged.com/onyx/rss/) that did the heavy lifting of parsing the RSS, converting it into an array of data I could then manipulate. IN fact, I found a whole collection of evolution of this code, look how simple the first version (may 5, 2003) was.

Sometime in the next year, Onyx either went away or I came across the more robust MagpieRSS parser which is what powers Feed2Js now, though Magpie has not been updated since 2005.

And Magpie is a bit of a problem, since it only works on well formed XML for the RSS feeds.

I have tried a few times to set up a version that uses the more versatile and modern SimplePie parser, but every time I try it, the folks from Modevia email and let me know the server process dies due to overloaded memory. I’ve tried all of the suggestions for dealing with SimplePie memory leaks but end of with the same issue.

Honestly, I am at the end of my technical skills here. I dont know how to write or fix a parser nor do I know what to do about Simplepie and the large load it must need to deal with all the feeds Feed2JS supports.

It really needs someone more code savvy than me. So I am asking will anyone

cc licensed flickr photo shared by KayVee.INC

Here’s my stipulations:

• Feed2JS must always be free and open to use, and sourc continued to be provided via the Google code site.
• While technically it could be done, no advertising or non feed content should be published in the output.
• Any updates must be done so in a manner that does not break any previous use. I have managed to do this for 7 years and lots of additions.
• I’d even hand over ownership of the Feed2JS.org domain.

There’s a lot that could be done to improve the front end, like a more AJAXy preview set up.

I have no great hopes that anyone would leap to take on an unfunded and uncommercial coding project, but I guess it never hurts to ask. I’m not in a place to do much with the code besides keep it as is.

Please?

cc licensed flickr photo shared by jeffreyw

If my stats are right, in the 6 hours since the cache was cleaned out (daily), 45,000 some unique feeds have been run through here, which is about 60 feeds per minute.

I hardly do anything to the code and I hope it does not blow up on anyone. There may have been some issues since the time I was trying to implement a new parser (which killed the server which for years has been kindly donated for free by the kind folks at Modevia Web Services)…. in restoring it I may have put a deprecated version of MagpieRSS; I just re-adjusted and the old FeedMachine seems to be working.

For the last few years I had the source code hosted at EduForge but as there were some rumblings of changes there, and that despite my immense depression over being Wave Rejected, I am a Google Fan Person— so I have moved the Feed2JS code to Google code at http://code.google.com/p/feed2js/.

I am still learning how to set things up there, and I don’t have source code established as check in / check out basis (like it matters, no one ever jumped at the chance to tinker in the code when offered). But since my contact info is not exactly front and center on Feed2JS site, I prefer if people use the Issues submission tool at the new site to let me know of any problems (no guarantee I can do much, but I will try)

What is still half baked in my lab is:

* a way for the users of the primary and soon to expand mirror sites, be able to select the mirror they wish to use. I am not running the service on the eduforge site, the primary public site to use to generate feeds should be http://feed2js.org and the 4 or so mirror sites I still have access to.
* a way for mirror sites to dl the site, and register their site as a mirror. So I don’t have to
* I have taken away completely the option that generates an HTML version of the content- this was primarily for the NOSCRIPT portion of the code, but it’s been abused by spammers to boost their Google count. Long story, but stupid spammers again waste my time. NOSCRIPT will simply display a message now and link to the RSS source (is there something else it should do??)
* I have some code to experiment with that might allow the cut and paste magic to filter out items defined by keywords passed to it. No promise.
* I hope someday, someone with better skills than me can find a way to do an AJAX approach for the feed previews.

Anyhow it is in slow progress, but moving a bit.

And lovely- the original site I hoisted at Maricopa is still a dead server, and I feel horrible that there are thousands of pages still ping-ing it. Sigh…

A few weeks ago I bought a domain, and with some hosting donated by Aaron at Modevia Web Services, the Feed2JS service that lives now at http://feed2js.org/ has its own home at http://feed2js.org/. All I’ve done is more or less move the current site in whole, made a few edits, and set it up for some stress test.

What does it mean? If you want to give it a try, this new server is going to be watched over and given technical support if needed. All you need to do is to either rebuild your cut and paste feed code or simply edit your JavaScript to replace the 2 occurrences of

http://feed2js.org/

with

http://feed2js.org/

I am not making any changes yet at the Maricopa server, but likely within a week or so, my access to it will vanish. If all goes well over the next few days, and it is time to throw the switch, I will most likely leave on the Maricopa site a web re-direct, so all requests to the old server will automatically be forwarded to the new.

You can still download the source code from the new site, which many people already have done– not to be charitable and be a public mirror, but just to run their own code, so their site is not hung out to dry if the remote server goes belly up. There are thousands of others that are running their own server. I have done it myself in a number of sites where having a local version makes sense.

This is an interim plan. I am still mulling over how to reconstruct the code logic to do the feed fetching asynchronously, and having options for designating alternative sites. Call it Feed2JS 2.0 if you need a buzzword. Anyone interested in helping wrangle the new code with me? I’m ready to pop the source code on a Forge site.

But first, let’s see what happens when 13,000 requests per day start swimming to the new site…

Don;t worry, I will leave the legacy… er, curent code as is. My own prime directive is not to break anyone’s previously constructed pasted JavaScript, so no parameters have changed name or purpose since 2004, or whenever it was I rolled this out.

But David’s comments hit home, that having a handful, 10, 20, 50 other sites one could use to generate a feed, were not really “mirrors” as I called them, suggesting if one doesn’t work, the code would tried the next… they are more like dumb clones. I re-called that I had done some test cases, and got working, a time out script, so if a server did not respond (e.g. like when the Maricopa site blinks out) it responds with a JavaScript alert, and an option to abort or keep on trying.

But it would get tricky to send the request to another site in this case, given the logic flow I had set up. But…

This would seem like the perfect time to roll out some AJAX implementation. This might mean that, JS code could seek out the server and parsed RSS content, and display it to a specified part of a page. But let’s say, it got no response… well, perhaps the JS could contain 3 or 5 alternative sites that ti could send the same parameter string to… then it could keep trying. And there should be some way to put a status indicator in the page’s place where the outpit goes.

I’ve only done minimal AJAX but it seems, like the right way to go. Anyone have any thoughts??

The downside is it would take a bit more cutting and pasting, as AJAX calls for a suite of JS code that needs to be in a web page’s content, or linked to a library somewhere. I worry about something that might be more robust functional wise, versus something that might be more challenging to use.

Help?

Bottom line- in less than a month, I will not even have access to the server at Maricopa where it sits. the folks there may look after it, but they do not even know much about it, and have other things to worry about. And despite some people’s “cute” attempts, it’s not part of my job at NMC to leap into action when the server borks, or help people figure out why their output is aligned left or…

So in some sense I want to let it out, let it go, or hopefully find some generous angel(s) to adopt it. I don;t want to own it, I want others to get free use out of it.

So I have half a plan. I have a company that has offered to host the main site for free. And there is even a parking spot for it. But I really want to find some other folks to chip in and help, some coders to tune up the code, some designers to keep the site up to date, some friendly folks to help people sort out issues. I want the code to be out there in open source land.

So likely, in the next week or two, a new site will appear, and the http re-directs will steer traffic from Maricopa. But I need some thinkers to help figure out a system that would allow people interested in being a public mirror (a copy site made available to others), to easily download the software, and then somehow register their new site. There needs to be a system that can update the other mirror sites that a new one is one the scene (my present awkward methods relies on an RSS feed from the home server- so if that goes down, so goes the list of mirrors).

So is anyone interested in helping out? I am going to put the software out there, but I am not going to be a lone support team for it…