cc licensed ( BY NC SD ) flickr photo shared by epc
In my last post, I quickly overviewed the wordpress customizations I did to set up the ETMOOC Blog Hub. Using the Feedwordpress plugin for a few feeds is easy to do, and it does a rather slick job of finding feeds from a blog URL.
The messy part is dealing with a lot of blog feeds. Getting this part right is more than just tossing URLs into a magic box, you have to have a good grasp of how RSS feeds work in different blogs.
It’s messy.
Because of those pesky humans.
Over at ds106 we have a rather elegant blog registration system that Martha Burtis designed, that actually does a web registration and automatically enters someone’s new blog into Feedwordpress.
The thing is there is a bit of variability to deal with when allowing people to bring in any blog platform (that is what we want), because it can eb confusing to the individual, especially if they are new to blogging, what we ask of them.
The thing is, it’s most easy if someone says, “I am going to do this ETMOOC thing, so I am going to make a new blog just for that stuff” – all we need is the blog URL and Feedwordpress can figure out the right TSS feed to use.
It gets more complicated when someone has an existing blog they want to use to do ETMOOC writing. There is nothing wrong with this approach (especially since it is mine!) but we don’t want to subscribe to everything the blog publishes- we just want posts that are related to ETMOOC. So the person with the blog has to decide (and understand how) to use tags/categories in their posts to mark things they want to syndicate.
This is quite a powerful concept that is easy to overlook – it means I can do things like use a single blog to selectively push content to different places through an understanding of the flow.
This is compounded by the different ways blog platforms are st up for this kind of syndication.
Messy.
cc licensed ( BY NC SD ) flickr photo shared by epc
Typically in Feedwordpress, you can enter the URL for a site, or a link to a category/tag archive, and it presents a choice of all the feeds it finds- sometimes it is as many as 4 or 5- there is the main site feed, a feed for comments, a feed for a tag or category, and often flavors to choose from (RSS or Atom). So as a site syndication wrangler, you have to know the difference in these feeds.
Alec and I decided to have people get us the information via a google form, and I offered to be the wrangler. We considered doing a screencast to help people figure out what URL they need to give us, but after scribbling down all of the variations, a video seemed not quite as useful, since there might be 8 different possible ways. I abhor screencasts where you have to wade through stuff that is most relavant.
Instead I set up a reference page on our site, How to Connect to the ETMOOC Blog Hub. The point is to emphasize again the difference between using a blog dedicated to ETMOOC (which is easiest for us and them) versus using a more general purpose blog. I tried to cover the possible platforms we thought people might use. To summarize the differences…
I did not want people to bother with figuring out where their RSS feed was, just to be able to provide a URL that points to the place we find their ETMOOC content, again a whole blog, or a category/tag/label archive within a blog.
- WordPress, the free hosted or the open source host yourself is.. the best. For one thing, the URLs for wordpress have a human intuitive structure. To find the URL for an RSS feed, you just slap “/feed” on the end of a URL (providing for self hosting they have done permalinks). If your blog is http://cheesydogsnacks.wordpress.com/ then your tag URL is (if you tag “etmooc”) http://cheesydogsnacks.wordpress.com/tag/etmooc and the category URL(if you use a category) is http://cheesydogsnacks.wordpress.com/category/etmooc. All of these feeds are “discoverable” meaning they are embedded into the source HTML so web tools can parse them out.
- Blogger is popular, but I cannot believe something smart people at Google work on is so wobbly. First of all, if you want to segregate out your ETMOOC posts, you use a “label” (just to be different they use something othe than tag, sigh). If my site is http://cheesydogsnacks.blogspot.com/ can you guess what the URL is for my ETMOOC Labeled archive or RSS Feed? Are you ready? You will find all of of my ETMOOC posts at … http://cheesydogsnacks.blogspot.com/search/label/etmmooc not really horrible, but look at what the RSS feed turns out to be– http://cheesydogsnacks.blogspot.com/feeds/posts/default/-/etmooc And worse, Blogger does not include this as a discoverable feed for that label archive page. That is just bad. Pee You.
- TypePad looks like it has equivalent feed constructs for tags and categories like WordPress Of course, I cannot recall the last time I saw a TypePad blog, but they can eb sued. Their tag/category feeds are also discoverable.
- Tumblr is so easy to create new ones, I cannot see a reason to consider using its tags It takes like 2 minutes to add a new tumblr blog. If your site is http://cheesydogsnacks.tumblr.com/ then your feed is easy, just tack rss on the end.
- Posterous too has tags, and and easy way to find its URLs. The URL for an ETMOOC set of posterous posts might be http://cheesydogsnacks.posterous.com/tag/etmooc — the RSS feed turns out to be http://cheesydogsnacks.posterous.com/rss.xml?tag=etmooc
- Weebly? I have no idea how it works. A bunch of teachers like Weebly; it has a category-like function, but I have yet to see a way to get an RSS feed for it.
Again, we are not asking people to figure out their feeds, but I need to know how.
You see, Feedwordpress has this nifty feature to add a batch of URLs, so you cna subscribe a bunch at a time, rather then one by one. So you enter the URLs:
The thing is that it will use the FIRST feed it finds at those URLs. If it is a Blogger URL, it wont find the feed for a label, just the entire blog, defeating the whole purpose of using labels. Even for WordPress, a tag/categpry feed is usually the fourth feed in the page, so the batch form will miss it.
So to be able to run these, you can make it work if you can pass it not the URL, but the feed itself.
Is anyone still with me? I don’t mind, I am writing this as much for my own records.
As the google form data comes in the spreadsheet, I actually end up having to check all the feeds, though I can recognize a wordpress category/tag url, so I just change it be adding /feed to the end. Tumblr URLs work. Blogger urls with labels, I have to do a little legwork to re-write them to that funny RSS form.
What I have been doing is colorcoding the spreadsheet cells for the status of the feeds.
Green is a url that was added ok. Green is good.
The other issues that come up with are people who have existing blogs who did not understand the part about ising a tag/category/label. So I have to mark them for more info (pink). Other ones have feed errors, or the persom who sent me their email address instead of a URL. (red) A few others have blogs that are private (kind of hard to be in an open course that way?)
Once I check all of the newly added ones, I can copy all of the green nes to Feedwordpress and add them in batch. Boom. Done.
For pink, I try to contact them via twitter (we asked them to provide their twitter handle) I put a little series of cells in the right side to construct this message:
The top cell is just a formula reference to the cell that contains their twitter handle, the middle one has the text of the tweet, and the bottom one uses CONCATENATE to string them together, so I can copy from there and paste to twitter:
@RosemaryFPowers thanks for sharing your #etmooc blog. Please send URL for a specific tag/category so we syndicate just etmooc content. thx.
— Alan Levine (@cogdog) January 12, 2013
I made another small formula to help construct the Blogger feed URLs:
The top cell is the “name” of their site (the part of the URL before ________.blogspot.com/ca/es). The second is the tag. And my formula can build that ugly URL.
I pretty much evolved this workflow today, and have processed 75 feed so far. We are running about 10% that need more information (tags, fix private settings, and one blog I spotted in the source that seems to have hacked content spilling into the feed and rendering it unusable).
Yes, this is a manual process with bits of automation. Frankly anything that is automated is made better by understanding how that automation works. Otherwise you are left to “it works / it does not work” and you just shrug and go back to Liking Facebook comments 🙁
I can live with the mess in this machine, and its actually been fun to work this out, and help people learn more about how their blog works.
cc licensed ( BY NC SD ) flickr photo shared by epc
Yes, a mess can be comfy.
@cogdog
For #etmooc blog hub: update for one.pulled.thread_1848@zoho.com (aka @RosemaryFPowers) URL = http://t.co/pxMd6Evt. thanks
I keep wondering how to handle registration in a better way. I have found people submitting their e-mail addresses instead of their url, too. Even worst, once I had a teacher submitting her postal address instead of her e-mail/blog. Like I said in my intro to #etmooc, mileage DO vary a lot when putting these ideas to work in other contexts.
Because of the preferences of many teachers, I provide a daily e-mail digest, which means they have to do one more registration. So I think it might make sense to have something like MailChimp verifying mail addresses and maybe URLs (something that a Google Form does not), and taking bringing them to FWP.
Something else that I’d love to see is a csv import option for FWP, so you can take a GDocs spreadsheet and import it right away, getting an error report that you can work quickly from. But I haven’t had the time/discipline/mojo? to work on it.
It’s messy, absolutely. And it would help to make it easier for the wrangler. Lots to do. 🙂
Hi Diego,
It’s nice to talk to a fellow feed wrangler. It seems so simple on the surface, but the details are many. We have a pretty good system on ds106 developed as a plugin by my colleague Martha Burtis that uses Gravity forms as the input. We dod some feed discovery to get closer; I think maybe a button to test/check/verify the URL might work. I am musing outlining a map of all the if / then statements one might need.
But there is a conceptual human gap as well. In a way I don;t mind it being manual, I get a chance to glance at a lot of blogs and as a community organizer it gives you a broad taste of the people entering your community.
It does get significantly easier of people are creating blogs solely for the purpose, early all platforms are easily discoverable for a site feed.
Oh, yes, that first “S’ in RSS? “Simple”?
The details are many indeed. I checked the ds106 plugin but I prefer to find some alternative that won’t involve cost (in case some other people want to try it out).
I do mind it being manual, because when I run into the same issue/error once and again, my patience gets tested. 🙂 Also, in my case, the twitter contact does not work, because most teachers are barely e-mail users. So I keep wondering whether this becomes way too complicated for people with limited tech skills, but then I think about what could become their first online learning experience (say, any LMS) and I get back to the URL checking… Perhaps we could do a xMOOC about URL checking!
I think the verify URL button is a very good idea. JS maybe? Looking for *blogspot* or *wordpress* in the URL, for starters (I had several people submitting the URL to their blogger profile instead of their blog)? I have PHP code that checks for RSS feeds given an URL, but I don’t know if you can do that with JS.
I often suggest people to go with blogger/wordpress because those are the most known platforms that provide RSS feeds for comments. If you want to aggregate comments you need those. I have yet to find that in tumblr/posterous.
Anyway what I’ve found, at least with the target I work with (keep in mind the context), is that this is harder than we think. Either people don’t read or this stuff is way too confusing for newbies. So I keep trying to make that first ‘S’ real. 🙂
https://getsatisfaction.com/weebly/topics/how_can_people_subscribe_to_my_weebly_site_via_rss
Towards the bottom (this thread spans _years_) a reply by Richard Huffaker mentions that in order to get an RSS feed from Weebly sites they need to create a blog.
Thanks, I have never weebled. I was able to get a weebly rss feed- though the only feed is site wide (there is no feature of tags/categories with RSS feeds)