In my last post, I quickly overviewed the wordpress customizations I did to set up the ETMOOC Blog Hub. Using the Feedwordpress plugin for a few feeds is easy to do, and it does a rather slick job of finding feeds from a blog URL.
The messy part is dealing with a lot of blog feeds. Getting this part right is more than just tossing URLs into a magic box, you have to have a good grasp of how RSS feeds work in different blogs.
Because of those pesky humans.
The thing is there is a bit of variability to deal with when allowing people to bring in any blog platform (that is what we want), because it can eb confusing to the individual, especially if they are new to blogging, what we ask of them.
The thing is, it’s most easy if someone says, “I am going to do this ETMOOC thing, so I am going to make a new blog just for that stuff” – all we need is the blog URL and Feedwordpress can figure out the right TSS feed to use.
It gets more complicated when someone has an existing blog they want to use to do ETMOOC writing. There is nothing wrong with this approach (especially since it is mine!) but we don’t want to subscribe to everything the blog publishes- we just want posts that are related to ETMOOC. So the person with the blog has to decide (and understand how) to use tags/categories in their posts to mark things they want to syndicate.
This is quite a powerful concept that is easy to overlook – it means I can do things like use a single blog to selectively push content to different places through an understanding of the flow.
This is compounded by the different ways blog platforms are st up for this kind of syndication.
Typically in Feedwordpress, you can enter the URL for a site, or a link to a category/tag archive, and it presents a choice of all the feeds it finds- sometimes it is as many as 4 or 5- there is the main site feed, a feed for comments, a feed for a tag or category, and often flavors to choose from (RSS or Atom). So as a site syndication wrangler, you have to know the difference in these feeds.
Alec and I decided to have people get us the information via a google form, and I offered to be the wrangler. We considered doing a screencast to help people figure out what URL they need to give us, but after scribbling down all of the variations, a video seemed not quite as useful, since there might be 8 different possible ways. I abhor screencasts where you have to wade through stuff that is most relavant.
Instead I set up a reference page on our site, How to Connect to the ETMOOC Blog Hub. The point is to emphasize again the difference between using a blog dedicated to ETMOOC (which is easiest for us and them) versus using a more general purpose blog. I tried to cover the possible platforms we thought people might use. To summarize the differences…
I did not want people to bother with figuring out where their RSS feed was, just to be able to provide a URL that points to the place we find their ETMOOC content, again a whole blog, or a category/tag/label archive within a blog.
- WordPress, the free hosted or the open source host yourself is.. the best. For one thing, the URLs for wordpress have a human intuitive structure. To find the URL for an RSS feed, you just slap “/feed” on the end of a URL (providing for self hosting they have done permalinks). If your blog is http://cheesydogsnacks.wordpress.com/ then your tag URL is (if you tag “etmooc”) http://cheesydogsnacks.wordpress.com/tag/etmooc and the category URL(if you use a category) is http://cheesydogsnacks.wordpress.com/category/etmooc. All of these feeds are “discoverable” meaning they are embedded into the source HTML so web tools can parse them out.
- Blogger is popular, but I cannot believe something smart people at Google work on is so wobbly. First of all, if you want to segregate out your ETMOOC posts, you use a “label” (just to be different they use something othe than tag, sigh). If my site is http://cheesydogsnacks.blogspot.com/ can you guess what the URL is for my ETMOOC Labeled archive or RSS Feed? Are you ready? You will find all of of my ETMOOC posts at … http://cheesydogsnacks.blogspot.com/search/label/etmmooc not really horrible, but look at what the RSS feed turns out to be– http://cheesydogsnacks.blogspot.com/feeds/posts/default/-/etmooc And worse, Blogger does not include this as a discoverable feed for that label archive page. That is just bad. Pee You.
- TypePad looks like it has equivalent feed constructs for tags and categories like WordPress Of course, I cannot recall the last time I saw a TypePad blog, but they can eb sued. Their tag/category feeds are also discoverable.
- Tumblr is so easy to create new ones, I cannot see a reason to consider using its tags It takes like 2 minutes to add a new tumblr blog. If your site is http://cheesydogsnacks.tumblr.com/ then your feed is easy, just tack rss on the end.
- Posterous too has tags, and and easy way to find its URLs. The URL for an ETMOOC set of posterous posts might be http://cheesydogsnacks.posterous.com/tag/etmooc — the RSS feed turns out to be http://cheesydogsnacks.posterous.com/rss.xml?tag=etmooc
- Weebly? I have no idea how it works. A bunch of teachers like Weebly; it has a category-like function, but I have yet to see a way to get an RSS feed for it.
Again, we are not asking people to figure out their feeds, but I need to know how.
You see, Feedwordpress has this nifty feature to add a batch of URLs, so you cna subscribe a bunch at a time, rather then one by one. So you enter the URLs:
The thing is that it will use the FIRST feed it finds at those URLs. If it is a Blogger URL, it wont find the feed for a label, just the entire blog, defeating the whole purpose of using labels. Even for WordPress, a tag/categpry feed is usually the fourth feed in the page, so the batch form will miss it.
So to be able to run these, you can make it work if you can pass it not the URL, but the feed itself.
Is anyone still with me? I don’t mind, I am writing this as much for my own records.
As the google form data comes in the spreadsheet, I actually end up having to check all the feeds, though I can recognize a wordpress category/tag url, so I just change it be adding /feed to the end. Tumblr URLs work. Blogger urls with labels, I have to do a little legwork to re-write them to that funny RSS form.
What I have been doing is colorcoding the spreadsheet cells for the status of the feeds.
Green is a url that was added ok. Green is good.
The other issues that come up with are people who have existing blogs who did not understand the part about ising a tag/category/label. So I have to mark them for more info (pink). Other ones have feed errors, or the persom who sent me their email address instead of a URL. (red) A few others have blogs that are private (kind of hard to be in an open course that way?)
Once I check all of the newly added ones, I can copy all of the green nes to Feedwordpress and add them in batch. Boom. Done.
For pink, I try to contact them via twitter (we asked them to provide their twitter handle) I put a little series of cells in the right side to construct this message:
The top cell is just a formula reference to the cell that contains their twitter handle, the middle one has the text of the tweet, and the bottom one uses CONCATENATE to string them together, so I can copy from there and paste to twitter:
— Alan Levine (@cogdog) January 12, 2013
I made another small formula to help construct the Blogger feed URLs:
The top cell is the “name” of their site (the part of the URL before ________.blogspot.com/ca/es). The second is the tag. And my formula can build that ugly URL.
I pretty much evolved this workflow today, and have processed 75 feed so far. We are running about 10% that need more information (tags, fix private settings, and one blog I spotted in the source that seems to have hacked content spilling into the feed and rendering it unusable).
Yes, this is a manual process with bits of automation. Frankly anything that is automated is made better by understanding how that automation works. Otherwise you are left to “it works / it does not work” and you just shrug and go back to Liking Facebook comments :-(
I can live with the mess in this machine, and its actually been fun to work this out, and help people learn more about how their blog works.
Yes, a mess can be comfy.
The post "Building the ETMOOC Blog Hub (part 2)" was originally pulled from under moldy cheese at the back of the fridge at CogDogBlog (http://cogdogblog.com/2013/01/building-etmooc-blog-hub-2/) on January 13, 2013.