creative commons licensed ( BY-NC-SA ) flickr photo shared by Tom Holbrook

Now that the Thought Vectors site is out and running (36 blogs syndicating in, 11 of them from registered students, not bad for the day before class starts), it’s time for a bit more of the techie WTF does that do post, in line with previous posts on such syndication hub sites.

WARNING: This post is long and full of jargon. I would like to explain it in a style to be universally understood, but life on this planet is only so long. And hey, did you think I was writing thix blog all these years for you? Silly reader, this writing is for me me ME ME.

How to Syndicate, Nate (Make a new Feed, Reed…)

In response to the idea that a flow of posts from a large community is more than anyone can digest, I’d been contemplating following up on Martin Hawksey’s brilliant work on the octel site and others to make something more like Google Reader (RI) (/me shakes fist and curses at Google, DAMN YOU ALL TO HELL). My interest in being able to do this clouded what Gardner Campbell reminded me, smacks head on table- a Feed Reader is not a beautiful site, one that leverages the media and payout of sites that get people interested in reading.

So less than two weeks ago, Tom Woodward and I got on a Google Hangout and started going through themes they already had available– they have a subscription to Woo Themes. I have a bit of a love/hate relationship with modern WordPress themes- they have come with a huge amount of overhead to make admin interfaces. I counted and one I used for another project had over 250 files in it- the crusty old Headspace theme here has like 35. They also can make it hard to do customization (let me tell you how inelegant Elegant themes are).

But I am using the Scrollider Woo theme for a client site, and have some good mojo there. We wanted something of a magazine style layout, something that was not just a vertical stack of posts, plus something that had a banner slideshow and portions that could be populated by different tag/categories.

We landed on the Editorial theme and it is the one we are running with now. I found the way it is constructed (parts of templates within “/includes” subdirectories) impossible to set up a child theme– for example:

forces paths to be a parent theme — why oh why do they not use get_stylesheet_uri() ?? So I have gone down the route of working directly off of the theme, which means they should not update it. (IMHO the desire to update themes is way over-rated, I am using one that is like 7 years old and it still works) (and it sure looks like an old theme) (my summer project is a new theme) (maybe).

In short, the Thought Vectors site was not all that different from the previous efforts in having Feed WordPress muscle in the feeds and re-organize them into different views.

But there is always something new in the mix.

The Feed Machine is On

I’ve gone toe to toe with Feed WordPress (FWP) many times, with its idiosyncrasies — all of its data storied in wp_links, the way feeds can go “bad”, the clunkiness of an un-paged admin interface, but after a few years of using FWP, we have an understanding.

My colleagues note the presence of Another Thing That May Be The Great Grail Shaped Hope for Easy Syndication– and it might be…

but I really do not have struggles with Feed WordPress that Tim Owens talks about. I do not doubt it can behave odd, but for the syndication sites I have done now six times– I think I have its number.

In many years of experience with RSS (Feed2JS is, ahem like 12 years old, and still going) I accept that RSS is really not really simple, it is messy. You get a lot of crap in feeds. Bad character sets. Weird markup. Missing data. Just thinking of the spectrum of RSS as one simple thing is… well too simple.

Let me repeat, RSS on a big scale gets messy, especially if you plan to accommodate any platform.

There are thing I wished were easier in FWP, but the experience that It Works (mostly) keeps me in the game. We have over 700 feeds pumping into ds106. Well worth following is the technology used by link farms. But I am not married to any tool, and am interested in poking around with the new kid.

But not for this site.

The harnessing of Feed WordPress, the Tao of Feed WordPress if you will, is understanding how to organize the flow. Otherwise, what you get is just a long train of posts.

creative commons licensed ( BY-SA ) flickr photo shared by cogdogblog

And this is the feature I am not sure I have seen in other tools, to attach categories and tags to incoming feeds. What I have done in every syndication project since ds106 is the following:

  • Convert all tags and categories on incoming feed content to TAGS on the syndication site. This means all of the ways people organize content on their own site, is put into a tag cloud, and allows for ad hoc organization (e.g. if you ask your community to tag or categorize all their posts related to Pink Hair Ponies as pinkhairpony then you can have a URL that dusplays all such syndicated posts).
  • Use Categories on the syndication site to organize content authored there. This means you can create a local structure for announcements, assignments, etc that does not conflict with syndicated stuff.
  • Use Feed WordPress to add categories and/or tags to incoming content. This is where you create a means to provide subset views of syndicated content, like a category for all open participants, or for students in a section, or for just bloggers you like. What this means is Feed WordPress can put additional tags/categories on incoming content, at a universal and at a per feed level.

I start by setting up the default FWP settings, under Categories & Posts, I set the options to move all incoming tags and categories to be tags inside the syndication site.


Then I also set up a category (“Vectorized”) AND a tag (“syndicated”) to be applied to every syndicated feed (just in case I need use down the line)


Just as a Way I Do Things, the “Vectorized” one is the parent category for all my syndicated content, and is also the default category in WordPress (Formerly Known as Uncategorized).

So now, I back up to describe the structure of the Thought Vectors community; every participant has a blog of their own. We have for blogs

  • Open Participants
  • Enrolled students in six sections of UNIV 200
  • Faculty who each teach one of those sections of UNIV 200
  • Staff or maybe even other students.faculty at VCU

I tried a go at gliffy to show the parts of the ecosystem

(click image to see full sized glory)

(click image to see full sized glory)

All of the pentagram shapes (5 sided like a baseball home plate if you forgot geometry) are people from VCU- UNIV 200 students (yellow), UNIV 200 Faculty (green), and others (blue). Purple circles are open participants.

RSS provides the content for everything copied into the site, as managed by Feed WordPress. All posts from Open participants are added to a category; all UNIV 200 students are in a category, and the same posts can be added to a category just for one section. The UNIV 200 instructors each have a personal blog for posting (in a category); plus the teacher of Section 008 has her posts also appearing in the category for section 8.

The structure I set up for Thoughtvectors uses category hierarchies, used in menus, in the theme, and in the footer– as listed


  • Vectorized (category added in FWP, all syndicated sites)
    • All Blogs (category added in FWP to all syndicated blogs)
      • Open Participants (category added in FWP to all syndicated blogs from people self identified as open participants)
      • VCU (category added in FWP to all syndicated blogs from people self identified as being affiliated with VCU)
      • UNIV 200 Faculty (category added in FWP to all syndicated blogs for faculty teaching UNIV 200)
      • UNIV 200 Students (category added in FWP to all syndicated blogs from students enrolled in one of the six sections)
      • Section 005 Ryan Cales (category added in FWP to all syndicated blogs from students and instructor for Section 005)
      • Section 006 Jessica Gordon (category added in FWP to all syndicated blogs from students and instructor for Section 006)
      • Section 007 Jason Coats (category added in FWP to all syndicated blogs from students and instructor for Section 007)
      • Section 008 Bonnie Boaz (category added in FWP to all syndicated blogs from students and instructor for Section 008)
      • Section 009 Gardner Campbell (category added in FWP to all syndicated blogs from students and instructor for Section 009)
      • Section 010 Jon Becker (category added in FWP to all syndicated blogs from students and instructor for Section 010)
      • VCU Faculty and Staff (category added in FWP to all syndicated blogs from people self identified as being affiliated with VCU beyond UNIV 200)

I also have Feed WordPress set up to add a “blogged” tag to all syndicated blog posts (since we are also bringing in some non blog information from twitter and diigo); this is used by the editorial theme to populate the top 4 recent items and the 6 ones that show up at the bottom.

I did all this manually to get the site set up; but now in operation this is all managed by a Gravity Form and a pile of duct taped code (see below).

Got Icons?

Because this is a theme that uses featured images, we deployed two plugins to make this featured images work. The trick is that for WordPress themes to display featured images, that nearly always have to be ones in the local install. Because we are syndicating in posts, all of the media live elsewhere.

Thus we use the Feed WordPress Advanced Filters plugin, which has a number of things that can help modify syndicated content, but the image filter is used to generate copies of all remote images as media in the local install. Once enabled, this plugin (actually a plugin that is a plugin to Feed WordPress) adds a setting to the FWP Posts and Links setting


Yes, we do end up with a ton of other media, but this also comes in handy down the line if the blog disappears, as another feature of Feed WordPress is unsubscribing and setting the content to point to local copy versions.

We also use the Auto Post Thumbnail plugin which makes a featured image out of the first image it finds, if one is not specified (although this is also a part of our Woo theme, not quite sure why I added it, but hey, everything works). There are a few shortcomings, like we have some themes that have invisible GIFs in the layout or sometimes the first one found is a facebook like icon from the sidebar.

This (ahem) might be an encouragement for people to use images in their posts! Ya think?

And the last thing that helped is that for the Woo Themes, under Theme Framework Options, I was able to define an icon to be used if no featured image was provided.

Feed the Others

Twitter is a key part of the open experience, and we’ve been lighting up the #thoughtvectors hashtag for weeks. We are running the Magically Delicious Twitter Tags Spreadsheet by [Sir?] Martin Hawksey, and I have them all embedded into local pages on the site– the most insanely useful one to me is the Twitter Explorer, where you can really try to make sense of the connections, and even view back one person’s conversational history.


I still think it is useful to also have the tweets inside the WordPress database, just in case down the line we figure a way to do some analysis or visualization from the tweets and blog posts together (they should be related by person and time).

So as I have done before, I use a labnol script that converts a hash tag search into RSS (shakes fist at twitter for nuking RSS feeds), so I can bring it into Feedwordpress. While there is not a big presence to show them on the site, I do use a technique I have done before to display tweets as tweets– the RSS feed gives you the text of the tweet as the title and the item, but all we need is the link to th tweet, and modify the template to use WordPress oembed function to display.

To manage this, all posts from the twitter feed are put in the Twitter category (the touch if Feed WordPress Category settings).

For example, in the archive.php template, we do a check on each item to see an item in the category (or it is the category itself), and rather than using the typical display we use the oembed function:

I ended up doing some styling bits and template nudging for the archive of the twitter category (again adding the oembed, but also making it full width (3 tweets wide), and changing the default number of posts per query loop to be a multiple of 3.

tweet archive

The other syndication source (for now) is diigo where we expect students to be marking up resources. My first impulse was the hope we could get an RSS feed site wide for everyone who used the thoughtvectors tag in diigo; alas the service does not provide that. I was dipped too much in delicious, which sad as it goes, I am not feeling all that trusting of anymore.

Our compromise is to get the feed from a diigo group we set up which we are syndicating directly in via

Considering the goal of the course, this is not a horrible compromise. I would think course participants ought to tag in their own diigo account and maybe set up a widget or syndicate to their own site. I see it as a worthy practice for people not to share everything they bookmark, but the best resources they find in the course of their research by sending them to the group.

Oh Feed, Where Art Thou?

A big part of the mix is how to get people’s blog feeds, this being the holiest of holy grails. Having cleaned up after submissions in the signup form at ds106, and having done it via collecting via a Google Form, I feel like a lone wolf in howling how messy this gets.

Let’s put it this way, if we are syndicating everything from a site, it is simple to write a script to grab a site wide RSS feed. That works (and we are using it).

But it gets an order of magnitude chaotic when you ask people to share a feed for a category or a tag on their site. If you know everyone is using WordPress or tumblr (as we do in Project Community), you can make uniform instructions. When feeds can be anything, this is what you have to deal with (and having hand checked over 600 feeds for ETMOOC I have some experience to speak from):

  • Blogger uses “labels” as something like a tag/category. Yet, in the source there is no discoverable META tag for the RSS feed.
  • And the same goes for WordPress. I defy you to find me where in the source the RSS feed for a Tag/Category feed exists- you get the RSS feed for the entire site or the comments.
  • And it’s not just a matter of saying to someone “Just find the URL for your category/tag archive and add ‘/feed’ like — this only works for blogs with permalink settings turned on from defaults. Many blogs have URLs for tag/categories like which means the feed URL is something like — Deep down I know one could find a way to sort this out from a URL… well its still messy.
  • As I just found out today, it also gets messed up when someone uses Feedburner, since all WordPress feed URLs (the ones I might describe how to figure out) get usurped by the general Feedburner feed, and we get all the content syndicated. I believe you can create a secondary Feedburner URL for a tag/category.
  • And freaky Safari as a browser, when you test a feed URL like resolves it as feed:// — and Feed WordPress knows not what to do with it.
  • And heck, people often do not even understand a URL and put gibberish in like http://Idontknowwhatthismeans, or leave off http:// or just drop the “:” or …

In my experience, Feed WordPress does best when you provide it the exact feed URL and do not rely on it to figure it out. The desire is to make this “easy” (cue Todd Conaway’s Easy Button).

But since this is an experience where we want to teach people how the web and networked systems work, I think its better that they get some understanding what RSS is, that the web the see in a browser is not the data that is sent from a server to your computer.

At first I imagined a flow chart, but in conversation with Tom (after making a horrible looking flow chart on a white board), we settled on something more like a choose your own adventure game, with a series of questions that take people down the path to find their feed… or as Tom does well, to make it into a metaphor, along with the subtle tangential hyperlinks:

Being of the web often involves weaving together streams of information. Knowing that information can flow like water is important conceptually as you think about how technology might augment your intellect.

The following pages will help guide you towards identifying your individual RSS stream so that it might become a tributary of the great information flow (or so your posts can be like apple pies for whales).

So when I click on the yes response to “I have a blog” button you get and animated affirmation.

i have a blog

The main split is for people who have a blog and want to syndicate in a portion, we spell out the various ways they should construct their feed. More or less they should find the URL to their tag/category, figure out how to append or add something to get the feed, URL, and above all else, TEST THE URL.

We do throw in the mix some detection for people who indicate they will use their entire blog, what I call the Magic Box


This is a similar approach we used on the ds106 signup — some code I played with, Martha Burtis and Martin Hawksey both refined, to find a feed from a blog main url.

This is a wordpress page with a modified template that allows to act like a normal web form, I have one field named “blogurl” (the text input for the URL), and do some error checking on the input.

The work is done by a function in my functions.php. What it does is to fetch the source HTML for the given URL via the curl function and uses some Xpath matching to try and find the patterns for a RSS2 or an Atom Feed.

It does make the assumption that the first feed found is the main feed for the blog.

The aim for our setup is to make sure people can find the proper RSS feed for their site. From out first 40 submissions, we seem to be successful about 85% of the time.

Getting My Feed into the Machine

And now the complicated part! Is anyone still out there?

If you have your feed figured out, then you can head to the Syndicate Here form, what we set up so let people add their feeds directly to Feed WordPress. It’s a slight extension for what Martha Burtis rigged up for ds106, using Gravity Forms to collect the info, and some functions to write the data to the WordPress links table.

In a perfectly coded world, I would be able to provide this to you as a nifty plugin, and maybe some day that will happen. But this was a case of get it done, because students needed to get this info before classes start (which is technically tonight).

When you add a feed to the site, it is associated with a WordPress user account, but the person filling out the form never knows it, nor needs to log into the site. We do it to associate a user with the feed. To make unique usernames, we ask for a twitter user name (besides we want people using twitter), and we use it as the wordpress username when processing the form.

The script adds a few more fields to the typical WordPress user profile, these provides extra fields we may (or may not) user later.

So when Giulia Forsythe signed up, a user account was created using her twitter handle as the user name:

profile contact field

Maybe down the line we might try some geolocation (thought “Canada” is pretty big). But the thing we needed was a way to create user accounts in our form processing function. I will break this down into segments. What we need to end up doing is creating a link in the WordPress collection, and writing data to the notes field in a specific format, e.g.

link fields

This data for Laura Gogia’s blog makes sure we have the RSS Feed, an association with a wordpress user account, the default “blogged” tag, and some assigned categories (one for All Blogs, one for being from VCU, and one for being a VCU Faculty/Staff).

The nifty thing is Gravity Forms can do logical display of fields; so if a person selects “open” as an affiliation, the Organizational Affiliation appears (since for VCU we know what it is); and if they indicate they are a UNIV 200 student, we can toggle open the radio buttons for sections.

First we set up a hook and a function to respond to when Gravity form ID 1 is submitted.

The global variable is used for calling the database directly, and the filters put aside are something needed to write to the notes field as needed.

Next we do some data setup. We create a variable to hold the data that represents the default Links category Feed WordPress uses (usually “Contributors”), and extract the category ID into $contrib_category. We store the value for the form (ID 4) and save it as $twittername (plus strip out a “@” just in case they did not read the instructions to leave that off)

Now we set up all of the variables we need to write a query to the database that will create a new WordPress user based on the form data

If $user_id is not a an error object, we can proceed, first by updating all of the user extra profiled feeds we created above. I’ve added a replace just in case their feed URL is in the feed:// format. This code is rather tightly tied to the gravity form field IDs (I guess I could re-write then to use the form field names, duh)

Next we have to assemble the string for the notes field that indicates the categories that should be added to all posts from this feed. The format us each one is represented in the form:

where XX is the ID number of the post category (and they will be later separated by “/n”). We do this by building an array of all the values needed. “{category#23}” is added to every feed (it is the All Blogs) category; then we test the form values to apply the Open Category ({category#24}) and also update the profile field for the value if the organization.

The logic continues to separate out the UNIV 200 students, and if that is indicated, the {category#xx}” values for the section are actually coded into the form (more Alan laziness).

Now we can build the rest of the string the notes field needs, again, it its particular format required by Feed WordPress- we do the author association with the new author id created above, we add a tag for all, and the categories we just assembled (using some PHP to join the array elements with ‘\n’ as separators).

And at this point we can call a function that creates the WordPress Link, and after that update the link with the data for the notes field (this has to be done by a mySQL custom query).

The last thing is a really useless error message if the WordPress user was never created.

And that is all there is to it. Easy, right? Bueller? Bueller?

Understanding how Feed WordPress runs from the links info is useful, especially because the function to change a link’s URL from the Feed WordPress interface never works (“are you sure you want to do this”) with no buttons. So fixing the feed URL is as simple as editing the field in the link.

Can I Have Some Butter With that Feedroll?

I like having the site, and the category archives displaying in the sidebar a list of all blogs in that category, a feed roll. I’ve modified code I did first for ds106 to provide a shortcode that does this for a sidebar widget.

It also generates the full list on the page that lists all blogs, just with this shortcode (23 is the ID for the category All Blogs):

all blogs

For the widget, I hoped to do one shortcode that would work on all category archives (on ds106, I need to make a custom one for each different group of blogs)- that shortcode is really easy

I like using the Visibility feature of the Jetpack plugin, which allows me to provide logic for the places a blogroll appears, in this case I just hide it if you are on the All Blog list above (it would be redundant)

visibility widget

The widget has two modes; on a category archive page, it will list all blogs in that category, for example the sidebar on the category for All UNIV 200 Students (for now) lists all blogs, and a count of how many combined posts there are

student blog roll

This will of course get too long (we have 120 students), so by setting the mode of the widget

We can get the functionality that it shows on non-categopry pages- an indication of how many syndicated blogs there are on the site, and a random subset of 10 blogs from that group:

all blog roll

This took some crazy logic I will not bore the 1 person who might actually be reading this. The way it works is that it figures out what category id it should look for, and runs a custom mySQL query looking for it in the Links Notes field in that pattern desdcribed above.

IF you are really manic, here is the entire code for the shortcode:

This function has some potential to be re-coded to be generalized enough to be a plugin.


Would You Like PHP With Your Fries?

I am losing tracks of the other mods I have made- I did a bit of custom CSS to override or hide the formatting (“display:none” is your friend).

Because the excerpts for all syndicated content comes from somewhere else, I added most every where a source credits text. This is done in the theme templates- by using the FWP function is_syndicated() indicates if an item in a WordPress Loop has entered via Feed WordPress, and if so we can insert the source name that is stored in a post’s custom fields:

WordPress themes like the one we are using to display teasers for posts use a chunk of content called the excerpt a feature built into WordPress. A problem with syndicated content is media will be dropped (e.g. from tumblr) or it has no excerpt (Blogger). So a typical WordPress theme uses this function

In WordPress the default length of an excerpt is 55 words- and in a layout that can vary widely. For a more balanced display, I prefer to generate an excerpt based on character count (and strip out all the HTML so we do not leave broken tags or weird formatting):

This way I can specific in my templates how big/small the excerpt should be using (the function get_the_excerpt() grabs whatever it considers an excerpt, for Blogger it is the whole shebang apparently)

Another new trick was a hack to manage the front page sliders- those all come from posts with a specific tag. I wanted to add to the banner an image and a link to the Twitter Explorer. But that is a page.

So what I set up is a way you can create as a post the data needed to generate the banner slide- a featured image, and some text for the excerpt. But there is a custom field you can edit called redirect

custom field

My template hack means that whenever you link to the Post (try it, it’s really just a holder for the slide content), you end up on the URL in the custom field .

This was an edit to the single.php template, which is used to display a single post. The code below comes first before anything else in the template:

Doing this was more of a “I wonder if we can do it”) but now I see how we can add a graphic image banner that can be linked to any of the UNIV 200 class sites; each section has their own class size for section specific information from their teacher (we seem to call them “Clubhouses”).

And Now for the Close

I wonder if this is the longest blog post ever hoisted here. Like I mentioned, much of this is for my own documentation- as usual I have rigged something together that may not be completely replicable elsewhere.

But look- we have a pretty gorgeous looking site. We are already syndicating in 43 blogs, 15 of which are students all brand new to WordPress.

And the work for them, and the teachers is just beginning! It’s going to be fun/interesting to see how this plays out. Hopefully worrying about the server does not keep Tom up at night.

Syndication sites are no cake walk. They will rarely be push-button magic box. But if you can get close to the machine and not burn your hair, you can do quite a bit.

If now, you know where to find me.

(Hunched over fixing my sloppy coding).


Thought Vectors Activate! (and Syndicate!)

If this kind of stuff has value, please support me by tossing a one time PayPal kibble or monthly on Patreon
Become a patron at Patreon!
Profile Picture for CogDog The Blog
An early 90s builder of web stuff and blogging Alan Levine barks at on web storytelling (#ds106 #4life), photography, bending WordPress, and serendipity in the infinite internet river. He thinks it's weird to write about himself in the third person. And he is 100% into the Fediverse (or tells himself so) Tooting as


  1. Ah! That is why I was sent on an adventure to find and test my feed! It worked. And I learnt something about backstage on the internet show! And I loved the gif of course.

    I did not read the whole thing, no surprise there. But I read enough to marvel at how little attention I pay to what makes the magic happen. i just moan when it is not smooth and forget that there are many Alans and Toms everywhere tackling this stuff to keep it going. I am glad you write this stuff up, it helps me understands its complexity even if I know that I will never have the kind of patience you both have to do it. Each time I give it time, I learn a little more and I am grateful for that.

    The site looks awesome and i love the community diagram.

    Thank you both for loving this stuff so much and sharing your workings with the rest of us. Even if you only wrote it for yourself! Who are you kidding? :O)

  2. Astonishing, wonderful, bravo. “Thanks” can never be enough. And to think this journey began for me with your face floating above the auditorium in New Orleans in 2005.

    And now I’m at a loss for words. Everything Mariana and Tom said goes for me, too. With love.

Leave a Reply

Your email address will not be published. Required fields are marked *