Your Flowing Data (YFD) is described by its creator, Nathan, as “a Twitter application that lets you collect data about yourself.” but that does not really capture the magic essence.

I stumbled here in one of those lovely incidents of web serendipity aka happy accidents. I was being interviewed last week by someone asking about emerging technologies, and I mentioned being interested visualizations of data. We started talking about great sites and tools- I mentioned Information Aesthetics and the interviewer mentioned another site called Flowing Data a blog about “Data and Visualization (subtitle “Strength in Numbers:).

It took about one glance and I was subscribing to the RSS feed, and it was scrolling down the bottom when I caught the link for Your Flowing Data (YFD).

So the ideas is that it is a site you can use to track data for things you do regularly, or might want to monitor over time. And the nifty trick is you use twitter Direct Messages to send data to your YFD account. Also smart is that it uses Twitter OAuth for this part and for authenticating your YFD account. You simply need to follow @yfd (so you can send direct messages)

On reading the guide, it suggested picking something that represents an action phrase like “ran 1 mile” “ate chocolate” “Watched Waterworld” (ew, that one will never be in my log). A key is using a consistent data input pattern,

I decided to aim for some health related activities. Since I test my blood sugar several times a day for my diabetes, this seemed like something that was do-able. After doing a test, all I needed to do was send a direct message to @yfd with a message of glucose XX where XX was the measured amount.

(I did mess up twice and sent a public tweet).

Now my blood glucose monitor already tracks data and actually transmits it by wireless to my insulin pump (which helps on calculating the amount of insulin to take at meal time), so actually I have automated data:

but here is the trick. To get the data off of the pump, I have to connect it to my PC (the software only works in Internet Explorer) and then the data is locked in some format where all I can is generate pre-designed reports as PDFs. I cant do anything with the data except look at it.

The data is there, it’s my data, my there are only limited things I can do with it. Its rigid.

SO yes, getting the data into the Flowing Data site is a tad tedious (typing direct messages), but the beauty is the open endedness of the choices you can make about what to record and how to use it. YFD does not define what I enter as only exercise data or movies watched or food ate– I can create my own taxonomy of action words and data types. YFD is neutral on the kinds fo data that go into it.

I also decided to track the miles I run (d yfd ran 12 miles– yeah I am dreaming!), bike (d yfd biked 25 miles — more dreaming!), and other exercise as I add (d yfd walked 3.5 miles… d yfd kayaked 6 miles). Actually I dont even need to include the “miles” if I assume any number I enter is in miles (or kilometers or cubits or nanometers). And on top of that, for any physical activity, I add the time spent in minutes (d yfd exercised 75 minutes).

If I just enter something directly, it references it to the current time, but you can also give a time when the activity occurred, like d yfd read War and Peace at 5:30am

The data in YFD is all private- I am the only one that can see my data, though I can share the results in ways I define.

My YFD home page on the web shows a snapsot of my recent activity:

There are several elegant ways to visualize all my activity, one by calendar that shows at a glancea long view (over a year) but allows me to quickly pull up data by day:

The Tree Map shows at a glance what your most frequent activities are, each one hyperlinked to filter the data by that activity

And I can also go right to my data logs, edit entries, and export data in tab delimited format (no API yet for data):

That gives you some flexibility in viewing data (and there are search terms on all these pages to narrow the scope), but you can also define your own “pages” where you can mix and match bits of your data like averages, sums over time, data by time, last data lists, etc each as “modules” that you can move about on a page like Google Gadgets– and these pages you can make private or public– so here is my Glucose Readings page:

http://your.flowingdata.com/cogdog/page/82/

And you can mis and match kidns of data, so I have an exercise page that has calculations on my total time exercising, and then data displays for my runs and walks, plus even some of the glucose stuff– http://your.flowingdata.com/cogdog/page/83/

Yes, the manual direct messaging for inputting data is not optimal- it would not take much for a real geek to figure out a way for my blood test gizmo to send a tweet for me– but imagine if all kinds of medical devices were recording data in real time, submitting it through the nets to places where doctors can see them at any time (not just an office visit) or be able to do more aggregate scanning of public health (which feeds into great ideas like crowdsourcing medical treatments at http://www.patientslikeme.com/).

I’ve still not done enough to get a feel for the value of this as a tracking tool over the long haul, but I am most excited about how open ended a system this is. You can define any kinds of reporting system that you can dream up- you just need to frame it in verbs (actions) and nouns (measurements). And being able to mix and match data in such an easy way to generate visuals feels really powerful. I still don’t have an inkling for what, but sometimes you don’t know the full potential of something til later.

Mmmm, yummy data. And visualization!

There’s something afoot abut data.

Whether it is Sir Tim’s vision of a Web of Data (or is it the W3C Linked Data?), there is something emerging with not only the availability of more data (heck even the government is on it http://www.data.gov) but more what we can do with it… Tom Coates was on this early noting that (in 2008 at least) 90% of twitter activity was not at twitter.com web site but through the APIs that allow data (small bits) to flow in and out.

It is something I marvel at on a daily basis trying to follow the tracks of Tony Hirst who weaves magic with data.

I’m toying with something I hope to blog about i a few days that is prying open a whole pack of neurons for me in terms of “potential for something big and cool I cannot yet formulate).

But in the meantime, I’ve had swirling around the older neurons and experience with seamless flow of not so earth shattering data in some not so obscure software.

cc licensed flickr photo shared by shifty eyes.

I’m talking about song lyrics.

If you have been living in a Google Proof cave, perhaps you are un-aware that you can search almost any song title, or lyric fragment in Google, and score a free transcript of your tune (even more so if you include +lyric in your search box) I imagine a lot of these are people manually typing in lyrics to a web site (??).

I typically do this when I am looking to be clever (or aim to be) and use some fragment of song lyrics in a blog post, or maybe even parody it. A few weeks back, I was motivates to leave a comment on Gardner Campbell’s post about his 5th anniversary of blogging.

Knowing and appreciating our mutual love of the music of the Who, I thought I’d try a parody of “Long Live Rock” to change the lyrics to “Long Live Blog”. Typically I would do the Google Dance (woah look at all those results).

But I also remembered downloaded a nifty tool for Mac OSX called iClip Lyrics (oh dear, it lookis like version 2.1 now requires a few $$ payment). If you launch this tool and iTunes together, as you play a song in iTunes, iClip Lyrics does searches among the lyric web sites and downloads the full song lyrics, often in less than 10 seconds:

If it does not find, it, the app has some search tools to hone in on lyrics for ore obscure songs. I’ve not tried it, but MiniLyrics might do something similar for Windows users.

This handy, and might have been enough, but the magic part is that iClip Lyrics inserts the found lyrics into iTunes (it is a tab under the Get Info) for any song, so without any effort on my part besides playing a song), it is filling in data in iTunes, kind of like a database entry:

And then what is cool is that the same data flows to my iPhone when synced, so the lyrics appear in the iPod app:

This is not all so amazing, except that it is seamless to me- the lyric data flows from one app to the other in a sensible manner.

And even more, if I find the lyrics “wrong”, in iClip Lyrics (or iTunes), I can edit the lyrics to my liking -maybe even to the version I left a a comment for Gardner:

Down at Mary Washington the scene was changing
Tech and Learning were pushing out A-rating
We weren’t the first unit to blog off campus
And find interest quickly to ramp up on us
These days it’s all tweets and Facebook
Blogs are dead they say
Long live blog

Long live blog, I read it every night
Long live blog, come on and subscribe
Long live blog, be it dead or alive

Data.. its about to get more fun. Data… it’s not just for statisticians. Data…

Much too late, I do have a better answer now, Geocommons which (sigh, why would any site put a description of their service as a freakin graphic! let me copy paste!):

delivers visual analytics through maps; enabling non-technical professionals to view multiple datasets, draw conclusions, make decisions and solve problems without traditional GIS overhead

More or less, by clicking, you can select from a library of different data sets, and then layer them on a map (There are ways of adding your own data to the Finder).

Some quick examples:

• Prevalence of Bike Commuters in Less Obese States http://maker.geocommons.com/maps/566: So does one cause the other??
• Wait Time At Polls, November 4, 2008 http://maker.geocommons.com/maps/1334: using data from Twitter Vote Report
• Number of Facebook Users in the US http://maker.geocommons.com/maps/83?page=1: Is it for city folk only?
• Freshmen are Criminals http://maker.geocommons.com/maps/352?page=1: Not a conclusion I would make- looks like someone else was playing with data!

I gave it the 20 minute spin this morning, creating a world map that attempts to conclude that countries in violent conflict don’t do well in the Olympics (duh, it is just an experiment):

http://maker.geocommons.com/maps/1475

There was another layer I had tossed in to map cities with the tallest skyscrapers, which was meaningless and made more clutter than interest.

I poked around the sets of data until I found two things that maybe were not directly associated. Data sets can be..data associated with place, but also include things like roadways and geographic boundaries (like school districts, maritime regions). There are display options for the way data is distributed and displayed (colors, circles or squares).

The data is tagged, and you can combine tags and keywords in the searches- the results are a bit tedious in that they are paged 4 at a time. It would also have been nice if once I had made base map on a world scale, if I could filter out data sets that were at a different scale (like crop yields in Iowa). The data is also spotty, in some places there are pages and pages of EPA data, but the geography boundary category only has 2 items. It would also be nice if the results had an embed code.

But this is minor compared to the kinds of things you can create by connecting data and maps, and to visualize relationships. You may not achieve the energy of Hans Rosling, but I see a lot of potential here, be it in learning how to look at data (and then research whether there is a real meaning or just accidental correlation). Or an activity to add your own geodata to the collection.

Mashing up data and maps is so easy a dog can do it!