TV Memex

Just in case you have not had your fill of the previous two code dumps for the workings of the VCU Thought Vectors site… heres more. I can’t stop tinkering.

Last weekend Jon Becker emailed and asked if there was “an easy way” to mix together all of the RSS feeds from his students blog into one feed that he could use as a sidebar widget. He had done it before with Yahoo Pipes.

Now for some inexplicable web service that generates no revenue, Pipes still works. I keep a list of Feed Mixer services at Feed2JS. But why rely on a third party when we have the means right here– yep, to every nail it’s WordPress.

In the last Google Hangout with Martin Hawksey we lamented the challenge of syndication comments in an open course– only WordPress provides a feed for comment activity.

But for VCU, almost all of the students are using either the Rampages (VCU hosted_) WordPress or In theory, I told the crew, we could make a site that syndicates in all the student blog comment feeds, and we could put them in categories by section. These in turn can provide an outward bound RSS feed for an entire section– which is really what Jon wanted.

In fact, Gardner Campbell had done that on his own for his section- for all students in his Clubhouse site he put their comment feeds into a second site that in turn generated an RSS feed for comments back on his site.

I suggested doing the same thing, but for all sections– if we did it right, we could not only provide the feeds to each section site, we would have all of the comment activity in one place.

Hence the Thought Vectors Comment Reader:


The AVH Extended Categories Widgets plugin provides on the sidebar a count of comments per section, and also an RSS Feed (these are just categories):

comments by section

One of my own stumbles was my own misunderstanding of authors for feed items. When you subscribe to a single blog for its posts, each item in the RSS feed (a post) has the same author (the dc:author item). Feed WordPress will generate a WordPress user account for that author, and associate all syndicated content to it.

But in a comment feed, the author of an item is the author of the comment, so what Feed WordPress sees for an author is whatever name the commenter put in the form. Now for the students, they are mostly logged into the Rampages site, so the comment form automatically associates a comment anywhere to the same “user”. We might have a way then to track student comment activity across blogs (or anyone else if they are consistent in filling out the comment form with the same name).

Thus adding the Authors Widget plugin makes for another widget that shows the most active commenters:


Pretty neat.

I’ve been stumped some on getting some code to generate a list of random posts, I bet any plugin is going to be hampered by WP Engines clamp down on the MySQL ORDER BY RAND() query.

One thing not in there is a quick association to match all of the comments to the feed it came from. I got in my head that there might be a way to generate a list of which posts generated the most comments. This would call for perhaps a gnarly custom query.

I was determined to figure this out.

So here’s the thing, all syndicated comments come in with a specific URL that points to an anchor in the page (where the #xx is a database ID for the comment), e.g. comments might come in with permalinks like

what I needed was a way to do a query that would count the occurrences of post permalinks up to the #comment-XX. StackExchange to the rescue! There is a text selector to select based on a substring:

select substring_index(columnname,'_',1)

to select all rows from where columnname matched UP to the first occurrence of “_”.

Here is the winning query:

$custom_query =  
	SELECT count(*) as cnt, substring_index(wp_posts.guid,'#',1) as purl, substring_index(wp_posts.post_title,'by',1) as title
			FROM wp_posts
				wp_posts.post_type = 'post' 
				AND wp_posts.post_status = 'publish'  
			GROUP BY purl order by cnt DESC LIMIT 0,30

count(*) gives me the number of times the same match occurs, where the thing being counted is the guid column (that contains the post permalink( UP to the first # in the url. I was even able to deploy the same trick to get the post title since all of the syndicated comments are titled something like “Comment on I Love Piggly Wiggly Vectors By Captain Kangaroo” to get the title out by removing everything including and after “by”.

(yikes, I just figured out I would chop the title that contained “By” in it).

So the page of top 30 commented posts is now up at

top posts

Now while excited about this, I do notice some discrepancies when I go back to the blogs where these come from. One is easy to account for since the counter listed on a WordPress site includes pingbacks, which are not syndicated.

But that does not make up for all the differences.

Another issue is how many comments we can syndicated at a time- an RSS feed for comments doe snot return every comment, but up to the last 10, that is the default setting for how many items appear in an RSS feed

most recent

We have a good situation where many student blogs are generating more than 10 comments an hour. But the feeds then are not capturing everything.

One route might be to ask all students to adjust their WordPress Reading settings to a higher number. I am hoping there is a programmatic way we might be able to bump the feed items for just the comment RSS feeds (there has to be a hook/action for that, right??). Ideally I could roll that into a plugin we might network activate on rampages, which would make it happen on all blogs.

Ahh I skipped an important step- getting all of the 100+ blog comment feeds into this site. My last gory code post included a script I made to generated a blog OPML file for each course section. With just some minor tweaking, i was able to make a second script that generates an RSS Feed for the comments from all those blogs– for WordPress, it is super easy; get the home URL of the blog (That comes right from the Links data), like and just to it to make

But I got even fancier- within the Notes that Feedwordpress writes in the links data as it syndicates, it pulls in a string that identifies the syndicating platform– all WordPress blogs provide feed/generator:, Blogger gives feed/generator: Blogger, tumblr does something like feed/generator: Tumblr (3.0; @cogdog).

So I was able to include in my query that pulls data from the Links info, to skip any blogs not identified as coming from WordPress.

Here is the code for the PHP script I can call to generate a Comment Feed OPML file

< ?php
/* ----------------------------------
Feedwordpress Comments Feed OPML Generator

a hack and a half method to generate opml feeds for comments
frmo blogs that are powered by wordpress
Alan Levine

------------------------------------- */

//--------  a little config goes a long way

$default_cat = 23; // id for default category
$pretty_title = 'Thought Vectors Comment OMPL Feed'; // base for title

// load wordpress functionality so we can do stuff
require( 'wp-load.php' );

// get the parameters from url parameters
// these should be set up in functions.php
$catid = ( isset( $_GET['group'] )) ? $_GET['group'] : $default_cat;

// keep a reference for the current category for an archive page
$mycat = get_category($catid);

$pretty_title .= ' for ' . $mycat->name;

//--------- set up db query
global 	$wpdb;

// custom query to get subscribed blogs from the links table
$custom_query =  
				wpl.link_name, wpl.link_url, wpl.link_rss
	FROM        $wpdb->links wpl,  $wpdb->postmeta wpm
	WHERE       wpm.meta_key='syndication_feed_id' AND 
				wpm.meta_value = wpl.link_id AND 
				wpl.link_notes LIKE '%%{category#" . $catid . "}%%' AND 
				wpl.link_notes LIKE '%%feed/generator:'
	ORDER BY    wpl.link_name ASC

// run run run that query
$feedblogs = $wpdb->get_results( $custom_query );

// bail if we got nothing
if ( count( $feedblogs ) == 0 ) die ( "No blogs found for '"  . $mycat->name . "'" );

// headers to generate downloaded XML file

header ('Content-disposition: attachment; filename=thoughtvectors-comments-' . $mycat->slug . '.opml');
header ("Content-Type:text/xml"); 

// start output
echo '< ?xml version="1.0" encoding="UTF-8"?>';
<opml version="1.0">
        <title>< ?php echo $pretty_title ?></title>
        <outline title="<?php echo $pretty_title ?>" text="< ?php echo $pretty_title ?>">

< ?php 
// output each item, 
foreach ( $feedblogs as $item ) 
 	echo "\t\t" . '<outline text="' . htmlspecialchars($item->link_name)   . ' Comments" title="' . htmlspecialchars($item->link_name)  . ' Comments" type="rss"
                xmlUrl="' . htmlspecialchars($item->link_url)  . '/comments/feed" htmlUrl="' . htmlspecialchars($item->link_url) . '"/>' . "\n";              
?>	    </outline>

Having an OPML for each section made it easy to import into Feed WordPress via the batch option (adding categories became a manual process, so be it).

A small thing I did just tonight was to modify the sidebar.php template so rather than display the site description, it provides a real time status for how many comments are fed in:


This was easy! A Matter of editing

<h2 class="site-description">< ?php echo esc_html( $description ); ?></h2>

to read

< ?php 
// get the info for all links associates with "Contributors" category, ID=2
$syndicated_feeds = get_term( 2, 'link_category');
<h2 class="site-description">As of < ?php echo date("M d, Y")?> 
there are < ?php echo wp_count_posts()->publish;?> comments 
syndicated from < ?php echo $syndicated_feeds->count?> 
UNIV 200 blogs</h2>

Can’t stop this dog from digging in the code bin! But there has to be some data value in trying to gather as much of the comment flow as we can.

By the way, the image I created to put on the site is a hopefully obvious not to Vannevar Bush, its a screen grab from a screen recorded animation of something created in Director (go old skool multimediaO to demonstrate how the thing worked, modified with screens from the Thoughtvectors site and a student blog

UPDATE Jun 20, 2014 Thanks to Stack Exchange I have the filter for changing the number of items in a comment feed. I tried to make a plugin that had options for setting it, but was getting nowhere; for now, here is a cheap way to hardwire the number:

< ?php
Plugin Name: Bigger Comment Feeds
Plugin URI:
Description: The default 10 items is not sufficient to collect comment feeds from an active blog; this plugin givrs you 50! And is stupidly hard coded until I get options to work
Author: Alan Levine
Version: 0.1
Author URI:

function get_bigger_comment_feeds_default() {
	//  defines how many items to put in the rss feed
	return (50);

// this filter changes the default limit for comment feeds
add_filter( 'comment_feed_limits', 'bigger_comment_feed_limits' );

// like Nike says...
function bigger_comment_feed_limits( $limit ) {
   	return 'LIMIT ' . get_bigger_comment_feeds_default();


Now if Tom does a network enable on this for, all blog comment feeds will have 50 items, rather than the default puny miserly 10.

The post "A Thought Vectors Comment Aggregator" was originally thawed from a previous ice age and melted at CogDogBlog ( on June 20, 2014.


  • Paul

    I love it. This is an awesome way to encourage interaction. I wonder though – Is there a way to highlight uncommented posts and get people to do something about it? The February is Album Writing Month ( site discourages zero-comment songs, or zongs, by highlighting them ( As I recall, a user’s dashboard also gave counts for comments given and zongs busted. It’s a way of trying to make sure everybody gets some love. Just a thought.

    • Alan Levine aka CogDog

      I totally agree- and that was a first response from Tom Woodward.

      It’s very tricky. FAWM can do this more easily because all of the content is on their site, so they can track comments high and none easily. One database, one query.

      First of all (as far as I know) WordPress is the only platform that provides RSS for comments, which is how this site was strung together. I could flip the query, and list the blog posts with the fewest comments, but it would be 1, not zero. Just thinking out loud, the way to get the zero comment posts would be to syndicate in all the posts, and do some sort of logical intersection to find ones that have not been syndicated into the comment reader.

      An interesting idea to mull. And beyond that, some blogs have zero comments because the content has not been seen enough, but there are some posts which maybe just do not need comments (let the audience sort that out?)

  • WordPress Comment Subscriptions by Category |

    […] you’re doing a mother blog and want to see all the student comments (like Allen did with the #thoughtvectors reader) but since we have students using their blogs for more than one class things get messy […]

Leave a Comment

All fields are required. Your email address will not be published.