d yfd found one awesome data tool

I’ve been mumbling in twitter (like anyone notices) about a very interesting data gathering/visualizing tool that rides the back coat tails of twitter in a clever way. I’ll spill the beans first, but stick around for the story and the after blog coffee, okay?

Your Flowing Data (YFD) is described by its creator, Nathan, as “a Twitter application that lets you collect data about yourself.” but that does not really capture the magic essence.

I stumbled here in one of those lovely incidents of web serendipity aka happy accidents. I was being interviewed last week by someone asking about emerging technologies, and I mentioned being interested visualizations of data. We started talking about great sites and tools- I mentioned Information Aesthetics and the interviewer mentioned another site called Flowing Data a blog about “Data and Visualization (subtitle “Strength in Numbers:).

It took about one glance and I was subscribing to the RSS feed, and it was scrolling down the bottom when I caught the link for Your Flowing Data (YFD).

So the ideas is that it is a site you can use to track data for things you do regularly, or might want to monitor over time. And the nifty trick is you use twitter Direct Messages to send data to your YFD account. Also smart is that it uses Twitter OAuth for this part and for authenticating your YFD account. You simply need to follow @yfd (so you can send direct messages)

On reading the guide, it suggested picking something that represents an action phrase like “ran 1 mile” “ate chocolate” “Watched Waterworld” (ew, that one will never be in my log). A key is using a consistent data input pattern,

I decided to aim for some health related activities. Since I test my blood sugar several times a day for my diabetes, this seemed like something that was do-able. After doing a test, all I needed to do was send a direct message to @yfd with a message of glucose XX where XX was the measured amount.

(I did mess up twice and sent a public tweet).

Now my blood glucose monitor already tracks data and actually transmits it by wireless to my insulin pump (which helps on calculating the amount of insulin to take at meal time), so actually I have automated data:

but here is the trick. To get the data off of the pump, I have to connect it to my PC (the software only works in Internet Explorer) and then the data is locked in some format where all I can is generate pre-designed reports as PDFs. I cant do anything with the data except look at it.

The data is there, it’s my data, my there are only limited things I can do with it. Its rigid.

SO yes, getting the data into the Flowing Data site is a tad tedious (typing direct messages), but the beauty is the open endedness of the choices you can make about what to record and how to use it. YFD does not define what I enter as only exercise data or movies watched or food ate– I can create my own taxonomy of action words and data types. YFD is neutral on the kinds fo data that go into it.

I also decided to track the miles I run (d yfd ran 12 miles– yeah I am dreaming!), bike (d yfd biked 25 miles — more dreaming!), and other exercise as I add (d yfd walked 3.5 miles… d yfd kayaked 6 miles). Actually I dont even need to include the “miles” if I assume any number I enter is in miles (or kilometers or cubits or nanometers). And on top of that, for any physical activity, I add the time spent in minutes (d yfd exercised 75 minutes).

If I just enter something directly, it references it to the current time, but you can also give a time when the activity occurred, like d yfd read War and Peace at 5:30am

The data in YFD is all private- I am the only one that can see my data, though I can share the results in ways I define.

My YFD home page on the web shows a snapsot of my recent activity:

There are several elegant ways to visualize all my activity, one by calendar that shows at a glancea long view (over a year) but allows me to quickly pull up data by day:

The Tree Map shows at a glance what your most frequent activities are, each one hyperlinked to filter the data by that activity

And I can also go right to my data logs, edit entries, and export data in tab delimited format (no API yet for data):

That gives you some flexibility in viewing data (and there are search terms on all these pages to narrow the scope), but you can also define your own “pages” where you can mix and match bits of your data like averages, sums over time, data by time, last data lists, etc each as “modules” that you can move about on a page like Google Gadgets– and these pages you can make private or public– so here is my Glucose Readings page:

http://your.flowingdata.com/cogdog/page/82/

And you can mis and match kidns of data, so I have an exercise page that has calculations on my total time exercising, and then data displays for my runs and walks, plus even some of the glucose stuff– http://your.flowingdata.com/cogdog/page/83/

Yes, the manual direct messaging for inputting data is not optimal- it would not take much for a real geek to figure out a way for my blood test gizmo to send a tweet for me– but imagine if all kinds of medical devices were recording data in real time, submitting it through the nets to places where doctors can see them at any time (not just an office visit) or be able to do more aggregate scanning of public health (which feeds into great ideas like crowdsourcing medical treatments at http://www.patientslikeme.com/).

I’ve still not done enough to get a feel for the value of this as a tracking tool over the long haul, but I am most excited about how open ended a system this is. You can define any kinds of reporting system that you can dream up- you just need to frame it in verbs (actions) and nouns (measurements). And being able to mix and match data in such an easy way to generate visuals feels really powerful. I still don’t have an inkling for what, but sometimes you don’t know the full potential of something til later.

Mmmm, yummy data. And visualization!

 Share this barking on social media

If this kind of stuff has value, please support me by tossing a one time PayPal kibble or monthly on Patreon