Your Flowing Data (YFD) is described by its creator, Nathan, as “a Twitter application that lets you collect data about yourself.” but that does not really capture the magic essence.

I stumbled here in one of those lovely incidents of web serendipity aka happy accidents. I was being interviewed last week by someone asking about emerging technologies, and I mentioned being interested visualizations of data. We started talking about great sites and tools- I mentioned Information Aesthetics and the interviewer mentioned another site called Flowing Data a blog about “Data and Visualization (subtitle “Strength in Numbers:).

It took about one glance and I was subscribing to the RSS feed, and it was scrolling down the bottom when I caught the link for Your Flowing Data (YFD).

So the ideas is that it is a site you can use to track data for things you do regularly, or might want to monitor over time. And the nifty trick is you use twitter Direct Messages to send data to your YFD account. Also smart is that it uses Twitter OAuth for this part and for authenticating your YFD account. You simply need to follow @yfd (so you can send direct messages)

On reading the guide, it suggested picking something that represents an action phrase like “ran 1 mile” “ate chocolate” “Watched Waterworld” (ew, that one will never be in my log). A key is using a consistent data input pattern,

I decided to aim for some health related activities. Since I test my blood sugar several times a day for my diabetes, this seemed like something that was do-able. After doing a test, all I needed to do was send a direct message to @yfd with a message of glucose XX where XX was the measured amount.

(I did mess up twice and sent a public tweet).

Now my blood glucose monitor already tracks data and actually transmits it by wireless to my insulin pump (which helps on calculating the amount of insulin to take at meal time), so actually I have automated data:

but here is the trick. To get the data off of the pump, I have to connect it to my PC (the software only works in Internet Explorer) and then the data is locked in some format where all I can is generate pre-designed reports as PDFs. I cant do anything with the data except look at it.

The data is there, it’s my data, my there are only limited things I can do with it. Its rigid.

SO yes, getting the data into the Flowing Data site is a tad tedious (typing direct messages), but the beauty is the open endedness of the choices you can make about what to record and how to use it. YFD does not define what I enter as only exercise data or movies watched or food ate– I can create my own taxonomy of action words and data types. YFD is neutral on the kinds fo data that go into it.

I also decided to track the miles I run (d yfd ran 12 miles– yeah I am dreaming!), bike (d yfd biked 25 miles — more dreaming!), and other exercise as I add (d yfd walked 3.5 miles… d yfd kayaked 6 miles). Actually I dont even need to include the “miles” if I assume any number I enter is in miles (or kilometers or cubits or nanometers). And on top of that, for any physical activity, I add the time spent in minutes (d yfd exercised 75 minutes).

If I just enter something directly, it references it to the current time, but you can also give a time when the activity occurred, like d yfd read War and Peace at 5:30am

The data in YFD is all private- I am the only one that can see my data, though I can share the results in ways I define.

My YFD home page on the web shows a snapsot of my recent activity:

There are several elegant ways to visualize all my activity, one by calendar that shows at a glancea long view (over a year) but allows me to quickly pull up data by day:

The Tree Map shows at a glance what your most frequent activities are, each one hyperlinked to filter the data by that activity

And I can also go right to my data logs, edit entries, and export data in tab delimited format (no API yet for data):

That gives you some flexibility in viewing data (and there are search terms on all these pages to narrow the scope), but you can also define your own “pages” where you can mix and match bits of your data like averages, sums over time, data by time, last data lists, etc each as “modules” that you can move about on a page like Google Gadgets– and these pages you can make private or public– so here is my Glucose Readings page:

http://your.flowingdata.com/cogdog/page/82/

And you can mis and match kidns of data, so I have an exercise page that has calculations on my total time exercising, and then data displays for my runs and walks, plus even some of the glucose stuff– http://your.flowingdata.com/cogdog/page/83/

Yes, the manual direct messaging for inputting data is not optimal- it would not take much for a real geek to figure out a way for my blood test gizmo to send a tweet for me– but imagine if all kinds of medical devices were recording data in real time, submitting it through the nets to places where doctors can see them at any time (not just an office visit) or be able to do more aggregate scanning of public health (which feeds into great ideas like crowdsourcing medical treatments at http://www.patientslikeme.com/).

I’ve still not done enough to get a feel for the value of this as a tracking tool over the long haul, but I am most excited about how open ended a system this is. You can define any kinds of reporting system that you can dream up- you just need to frame it in verbs (actions) and nouns (measurements). And being able to mix and match data in such an easy way to generate visuals feels really powerful. I still don’t have an inkling for what, but sometimes you don’t know the full potential of something til later.

Mmmm, yummy data. And visualization!

Besides almost every post of unbelievable wizardry and in depth explanation of Tony Hirst, not often are there things that just knock me over breathless. Maybe I have been drinking the Web 2.0 Koolaid for too long.

I had one of those “wow” moments tonight with something that came out of the blue. I’ll share it all, and am curious if my excitement is misplaced or not (would not be the first time)

Like many others, I have had my “Wordle Moment” which is another example of this phenomena. The stunning visualization effects of making word clouds out of plain text is nothing short of astounding, and it not your grandfather’s tag cloud.

Beyond a few plays with it like many people do, it’s been more fun than function (nothing wrong). I did use it just yesterday to create a graphic banner for a new site related to tagging. But I thought Wordle took it up a notch when I saw how you could have it generate a cloud based on an RSS feed:

The thing tht caught my eye my there was the prominence of “Tigger” over “Eyeore” because in that week I had one post about these characters and two more piggy pack posts echoing.

But that was also the last time I ventured down to Pooh Corner in this blog, which prompted my to ponder in that post:

The uber nifty Wordle tag cloud generator now can take any URL that has an RSS feed and generate one of those lovely word maps of content.

Here is what the latest blabber from CogDogBlog has in it- heavily weighted by my serious examination of the Pooh- Eeyore Debate.

What would be cool is to grab a time series of these to see how word use changes over time.

And way down in the comments, Jason Priem mentioned he was working on some code to generate visualizations of tag clouds over time.

Well he just followed up and shared this amazing app/site- FeedVis- which watches a list of sites with RSS feeds (via an OPML file) and tracks the pattern of word use over time:

or how his site describes it:

I liked the idea of making a word cloud; however, though there were several options, I wanted more. A plain tagcloud from a feed is static: all the words happen now. Conversations change, and I wanted to see that. I wanted more context, too–information about who was using the words, and how. This is what I came up with.

The animation shows frequencies changing; color indicates whether a word in the selection is more or less popular than usual. (check out “election” in early November, for instance).

You can compare pretty much anything (a time period, a certain blog, all blogs, all times) with anything else. Most stuff also has more information behind it if you mouse over.

The example he uses is a list of top 50 edublogs Scott McCleod listed from Technorati. Once it loads (there is about 45 seconds or more to do this), you see an aggregated view of tags used by all over 50 blogs htat you can see in different time slices; maybe there are trends there.

Or you can examine blog in particular… say mine! and see how my use of words changes over time:

And is more than just seeing tags change in size or color; when you mouse over one, it gives a measure of your use of a word compare to the aggregated whole:

And then I can compare my tags to others in the same time period, say Dean Shareski:

or David Wiley:

And more than just this coolness- Jason is giving away the code! You do need your own server to run this, and best with an ability to run cron scripts, but am already thinking that this would be a great way to assemble the feeds we collect on emerging technology for the NMC Horizon Report.

I am just intrigued about this approach to looking at patters over time- a movie is often better than a snapshot.

Is it just me or is this way way way cool?

FeedVis rocks! Thanks Jason…