How many more times do people have to get stiffed by a free web service that just bites the dust and leaves you bubkas?

What follows is a monster post. You will find me ranting ar companies like Storify who offer free services that leverage our efforts building content and then they develop enough value to get sold. After the sale they just yank our content. I also offer an approach for local archiving your storify dying content, via a new home spun tool for extracting all embeddable content links and how to use it to create your own archives in WordPress.

On we go.

Storify Is Nuking, for no credible reason, All Your Content

Like many people, I caught the word via twitter yesterday that Storify was turning off it’s lights in May, and not letting anyone new in. I think they way they told people was putting a notice on their site. Oh, they sent an email 2 days later.

Okay there are two kinds of people or organizations that create things for the web. One is looking to make money or fame and cares not what happens once they get either (or none and go back to flipping burgers). The other has an understanding and care for the history and future of the web, and makes every effort to make archived content live on, to not leave trails of dead links.

Storify is Type 1. After getting enough of a value based on the free labor of grunts like you and me who built content in it, they got bought by Adobe, and swept up into some enterprise product.

I spent time paging through the Adobe Experience Manager Livefyre site without any understanding through all the pretty photos what the heck it was (it does have marketing-cloud in the URL). Oh here it is:

Adobe Experience Manager Livefyre: User Generated Content (UGC) Platform
Turn your audience into a content machine.

Gaack

They are going to orphan all the Storify content they once hosted, leaving black web site not found holes in the fabric of the web. They took a cue from Twitter mothballing Posterous. And many more.

The second kind of people? Look no further than IT Conversations:

For any ed tech greybeards out there, IT Conversations was one of the most valuable and community facing resources for the early web, one of the early podcast networks. They are very much Type 2. When they closed, did they just unplug all the servers, and yank all the content? No, they took care of the it:

The remaining assets of the Conversations Network (cash and intellectual property) will be acquired by the Internet Archive, another U.S. 501(c)(3) non-profit organization. All existing programs will be moved to the Internet Archive where the world will be able to continue to listen to them for free.

We hope to preserve all existing URLs by running a “redirection server” for many years if not permanently.

Those are people who understand and care about past, present, and future.

Storify? They are just yanking it all.

They do offer you an ability to export your Storify content, which, to me is not an archive at all. And it’s ugly. And all embeds of content, all links, shall forever go on the Great Pile of Dead Links.

The “export” is a static HTML version of your content, not how it looks on their site, which you have to save as a package of HTML and a wacksock of other crap, a lot not even needed. Your export includes a pile of Facebook tracking code and other add server links. I spent a chunk of time this morning cleaning one. That’s another post.

So now there is the usual flurry of suggestions to do things like use Twitter Moments or some other third party curation site. Ian Milligan asked the right questions and suggested taking on a reclaim your stuff mantra:

It’s not as bad as some platform shutdowns – there is notice and at least you can export your own content (one story at a time) – but it’s still a reminder of how vulnerable user-generated content can be online.

The first hot take is that it underscores how when we spoke about “archiving” a hashtag on Storify, we weren’t really doing anything resembling that term. We were temporarily parking it on a free site until it inevitably closed.

The second big point, to me, is to use this as an opportunity around how we could move forward as a field.

The thing is everything Ian uses as examples, and many others writing about Storify, is storifying just tweets. And frankly that seems to be about 90% of the uses I have seen over the last few years. Just a way to package tweets together. If that’s the need, then maybe Twitter Moments are the way to go.

But to me the whole value of storify was not for stringing tweets together, but mixing content from multiple kind of sites- Youtube/vimeo video, Soundcloud Audio, Prezi, Web sites, Slideshare.

Screenshot of a storify (link soon to be unavailable) showing tweets, web sites, vimeo videos

Not only that, you could insert headings and text sections to act as ways to augment or explain the content. Storify was way more than a bucket of conference tweets. And as Matt Croslin pointed out elsewhere in a Slack discussion, it’s not about the archive or embedding, the real advantage of Storify was it made it convenient to pull content from many social media services into one drag and drop interface.

What’s really lost is the convenience factor. It’s not impossible to curate mixed social media, and if you happen to be among the people that use a WordPress platform (hosted or self-hosted), you can do it in a place that some 3rd party profiteers will never yank from you.

I’ve Got Your Alternative Right Here

One person worrying about this was Robin Derosa:

https://twitter.com/actualham/status/940761091814117377

She had designed part of her class around using Storify. He Big Tech (or not so Big Ones), if you are going to radically change your platform, how about asking your users first? How about giving them some advanced notice? Is that too much to ask?

https://twitter.com/actualham/status/940765216891170817

People are chiming in suggestions. I had a thought:

Because… well, you can see it right above. WordPress has, and has had for a while, the ability to automatically embed social media content by simply putting the URL for a tweet, or the url of a youtube/vimeo video, the urls of a flickr photo, the url of a SoundCloud track as plain text on a blank line in the WordPress editor.

But it’s better to show, than tell. I took a bunch of tweet URLs in this thread, and a few more, and put them in a text file:

I was working on a page on the blog here, but it was slow ressponding, so I jumped over to a WordPress.com blog I use for just testing, and quickly published a post with this alt Storify approach. I then tweeted it for Robin (and can see that I used the wrong URL in the tweet, that’s the edit link, sigh)

Eventually I got the page published here, same embed content, different web site.

The building part is a bit more manual, cutting and pasting URLs (the embeds will appear in the visual editor, if you need to change the order, the HTML editor to me is easier to work with). And in a blog post, you might consider adding more context around what you curate, instead of just stringing together because it’s easy.

But at least it’s on a web site you manage, not some third party shady motel like Storify.

Down the Coding Hole, Emerging with a Storify Link Extractor Tool

An extractor tool to get your WordPress embeddable links out of the dying Storify. Based on public domain pixabay image https://pixabay.com/en/tongs-steel-blacksmith-utensil-24241/ plus the Storify logo, and some text I added

Storify’s last offer to you are some instructions on how to archive your existing content. They offer no answers to what you do with it, and they leave off the fact that they will unplug the server and break all external links (remember they are People of The Internet Type 1) (it would take very little effort for them to get their content in the Internet Archive) (Also remember this is Adobe’s decision, the company that bought Storify).

I found a simpler way to get to your archive than their instructions (which sound like it is actually whirring away like an archiver machine, it is not). Just take the url for any existing storify like http://storifried.cogdogblog.com/netnarr-meme-and-gif-a-palooza and add .html to the url, load that url http://storifried.cogdogblog.com/netnarr-meme-and-gif-a-palooza.html. There is your “export”.

To save it, you should use File -> Save Page As and use the option Web Page, Complete (in Chrome or Firefox, not Safari which gives you some unusable file). This will download an html file and a directory full of other “stuff” to enable it to run as stand alone web page. You can even run it off of your desktop machine.

But I suggest that you think like an archivist and how to organize your stuff. I created a directory in my local hard drive and call it whatever you like storifyarchive or storifysucks — mine is called storifried. Ha.

Then… for the example above, I create a sub-directory with the exact same name as the url or netnarr-meme-and-gif-a-palooza. Stand by for the reason. Now, while viewing that archive above, I do the Save Page As, and I save it as a file named index.html inside the directory above. All the “extras” get saved in a sub directory named index_files.

If you load index.html in a web browser you will see a copy of the same web page you see hosted on storify (well for the next 5 months it will be there).

Eventually, you will create an archive that mimics the url structure on storify, on directory for each one you archive.

If you have a web site, you can put all of this in a subdirectory in your own domain (no luck if you think a domain is a WordPress.com web site). Or, you can create a subdomain, mine is http://storifried.cogdogblog.com

I spent a lot of time Friday pouring over the “archive”. It’s full of facebook tracking code, charbeat.js ad server stuff. It also is rather plain and ugly, this is the exported version of the storify above:

It contains links at the top that point to Storify (which will be dead soon). And it’s pretty ugly, and looks not quite like what I created.

So I spent even more time inspecting HTML, comparing the source code of the online version to the archive, stripping out un-necessary tracking code, to produce my own flavor of an archive:

I’m leaving it to a followup post for how one might take this on (warning IT INVOLVES INSPECTING SOURCE CODE AND EDITING HTML, if that scares you go run shrieking/liking into the Facebook Hills).

But while looking at that source code I noticed a pattern of each Storify piece of content in the time line, it has a regular pattern (warning HTML code ahead, I trimmed some out because it’s long) where with a vimeo item and a twitter one:

Everything is a list item <li> and each has a data item like <code>data-permalink="http://twitter.com/grantpotter/status/248269509868257281" which is the exact URL that WordPress should be able to autoembed.

The wheels are turning, maybe it’s possible to create a tool that will extract all these links. It took a bit more iterating; I was able to create a tool where you paste in the entire source HTML and it extracted, using regular expressions (FREAK OUT) to grab these URLs.

But then I noticed that there are other kinds of elements in the mix, the ones where I added a heading or text to organize my storify content (I might be among the only people who used this). These look like:

I was first able to skip these in my loop that looks at all matches; if it has something like data-permalink="http://storify.com/.... Those URLs are useless. But I kept thinking there must be some way to get the value of what is in there. This took a lot of banging on the pipes (due to my weak regex skills), but I finally got it. A key was pre-preprocessing the raw html by adding line break characters after all </div></div></li> which makes searching easier.

So… here it is my first pre-alpha Storify Link Extractor Tool.

My suggestion is to first save your local copy of the web archive as described above. I’ll walk through with a new example, this is one I created as some storytelling for a night I experiences stranded at Chicago O’Hare airport, so first I go to the Storify archive url https://storify.com/cogdog/nightatord.html

I then create in my local desktop archive a directory named nightatord. From the archive link above, I do the Save Page As and use the name index.html inside that new directory (this is just preventative stuff, not critical to using my tool).

Now I go back to the Storify “export” https://storify.com/cogdog/nightatord.html. I am going to use my browser’s feature to view source (this varies in different browsers, use this guide for your set up, as a Type 2 Web Creator, this is a critical skill). I am going to do a command A (select all) and command C (copy). Grab all that HTML.

Now, go to the Storify Embeddable Link Extractor tool https://cogdog.github.io/storify-extractor/, and paste that big whack pile of HTML into the first form field. Click the Extract Links button

This one found 68 embeddable links, and no text blocks.

Results of using the link extractor tool

You can put these in a WordPress post, maybe organize them by a category or tag. My approach is to use the Page structure. I have a Parent Page at http://cogdogblog.com/storifry/ I will use eventually as an index, and each storify I archive is a child of this page, so I create a new page, title it and make the url the same as the original, then paste in all those extracted links.

Since I like making it pretty, I will add a featured image, the title intro text, and maybe a note that this is an archive. And here it is

I plan to do this for any of my storify content worth saving – make a local copy for future consideration, extract my URLs, and publish in WordPress.

A few caveats:

  • You might not want to do a monster storify. Mine work okay as I run a cache plugin.
  • So it might make sense to prune down the links needed to the most critical.
  • The one thing WordPress does not embed are link to sites, so you may want to edit those out. Or turn them into links

Not everyone sees their storify content as worth saving. Many of mine are not. But quite a few are used as references or embedded into other sites beyond my own.

Beyond all this gory code, HTML stuff, is a consideration of what kind of Web Creator you are, Type 1, meaning make and destroy, or Type 2, consider for the past, present and future.

I am for sure, a Type 2.

Update: Dec 20, 2017

Via a tweet thread I was looped in by Bruno Winck, you can always jump to Yet Another Third Party Hotel

Look at all the pretty pictures and storytelling in their about page

At university, while researching for his degree, our founder Jamil struggled to keep track of everything he found. He ended up grouping together useful links, images and notes into sections of a Word document, scrolling through page after page to find a single URL and wasting valuable time.

Jamil never heard of bookmarking services like pinboard, diigo, or others that have been around since mid 1990s? Poor Jamil.

After a year travelling in New Zealand, Jamil’s friends kept asking him where he’d been and what he’d done, and each time he had to paste lists of links into emails. He’d label them, but without any context or visual cues, all they ended up with was a wall of URLs and no clear structure. Despite his efforts, Jamil’s emails weren’t useful, and as he exchanged more and more emails, they became very difficult to find and consume.

I’m glad Jamil got to spend a year in New Zealand. It’s been one of my favorite places in the world. Maybe he has heard of these things called “blogs” that provide the exact need described here? Why is he multiple emailing links to people? If he blogged, he could write once, and share links often.

Heck I did in 2000 when I visited New Zealand. I created a blog before there was software, handcrafted out of HTML.

Sure, go move your storify content to wakelet and call me when they go south.


Featured Image: End Coal Now action flickr photo by Reclaim the Power shared under a Creative Commons (BY) license

If this kind of stuff has value, please support me by tossing a one time PayPal kibble or monthly on Patreon
Become a patron at Patreon!
Profile Picture for CogDog The Blog
An early 90s builder of web stuff and blogging Alan Levine barks at CogDogBlog.com on web storytelling (#ds106 #4life), photography, bending WordPress, and serendipity in the infinite internet river. He thinks it's weird to write about himself in the third person. And he is 100% into the Fediverse (or tells himself so) Tooting as @cogdog@cosocial.ca

Comments

  1. Postscripts:

    1. Thank God for the Internet Archive. Has Brewster Kahle gotten his Nobel Prize yet? Or his MacArthur grant? Or his Presidential Medal of Freedom? He deserves all of them.

    2. You know that Flickr falls in this camp as well. So far it’s still there, but it has no great export tools, and it’s on the level of a national treasure, and it’s owned by people who are not what I’d call Type 2. So I worry. And I don’t use it as much as I used to. And that’s sad.

    3. I keep hoping Delicious will actually make some kind of export tool available. Or that someone will make one (yes, a hint). I’m sure my bookmarks are full of link rot, but they also contain a valuable set of associative trails that I believe will be valuable to me in the long run. Delicious was one of my first Web 2.0 loves and the first Web 2.0 freebie to break my heart.

    Moral of the story for me: it doesn’t have to be this way. But try to tell that to the edtech industry.

    Gotta drink more coffee. My head’s getting dark inside.

      1. That’s exactly what I did, Pinboard has all my links going back to 2006, like 11,000 in total. I like also how it displays a small red 500 next to sites not responding. Pretty sure that Maciej will have in place a means to mmve del.icio.us accounts over.

        Pinboard is the bomb, it has compound tag links and feeds galore.

      2. Ah, a small element of faith is restored–the export function does work, just as you say. And I will be moving to Pinboard.

        I miss Delicious. An admirable service. Clean, simple, and founded on a love of wordplay and associative trails. I signed on at Faculty Academy 2005, during a workshop with Bryan Alexander. The world looked different then. I try to keep the faith but it ain’t easy.

    1. I could not agree more about The Internet Archive, I shudder to think what this space would look like without it. Brewster Kahle deserves All the Medals. I just donated $50.

      However I don’t see flickr in the camp. The problem is relying on it to be your archive. One should archive first, then upload. Ever since the NMC photo workshop in 2008 I shifted to a strategy of organizing my photos locally, titling, tagging, etc in my own local backed up hard drives. I once had an account on backify that previously offered a free backup service. When *they* changed (biting the dust), I was offered access to my archive. It was too big to handle normally, someone produced one on Amazon cloud, it was 80 Gb.

      Yes, I am not too confident of its owners. But I am not so sure that means it will be shuttered. Here is a small fact worth considering- the Flickr API has never changed or broken previous functionality. Google, Twitter, eta all have all done things to break their own API. It’s coming up on 14 years old… how many web services have stood that long?

      Besides, I have a lunch bet riding on flickr’s survival! http://pechaflickr.net/bet/

      1. I hope you’re right about Flickr, but I mean Yahoo–oh well. I hope you win your bet but I will not be putting any money on it myself.

        And the “audience generated content” is such a big part of what makes Flickr valuable. All the links, groups, commentary, annotations (though I haven’t tried that in awhile–maybe it still works).

        The real problem in my case is lack of confidence, which means I just don’t use it as much as I did. That deprives me of all the Flickr goodness, which is still there. It doesn’t help the service, either. But when the banks start failing, I start keeping stuff in my mattress.

        Speaking of which, do you have a photo mattress? You have strange dreams, but they’re full of interestingness (a term Flickr trademarked).

        1. Actually you mean Verizon, they actually bought Yahoo and remixing it with AOL to be “Oath” (ugh) https://www.nytimes.com/2017/04/03/technology/verizon-oath-yahoo-aol.html

          I’ve not seen any flickr export that preserves all those things. That may have to be left behind.

          Flickr notes came back last year, the data was always there in the API http://cogdogblog.com/2016/06/party-on-flickr/

          My flickr mattresses are a few hard drives.

          1. I keep reading Oath as Oauth.

            I’ve got a Flickr PHP export thing built out (or did a year or two ago). I have some hard drives as well but they’re scattered in a way that scares me.

          2. I have over 60000 photos on Flickr (and Google Photos) but like Alan these are my public (and most private as backup) views on the archive I have organized (and backed up) on my own drives.

            Great work here Alan as always. You are the example that I show my students about producing and sharing work with the why, the how, the what. Keep rocking.

  2. I wonder how hard it’d be to SPLOT something up that’s a bit closer to Storify.

    I never really used Storify but seems like the Twitter API search is the main feature and wouldn’t be too much hassle.

    Instagram would be a pain and not worth it to me . . . but maybe a few other things.

  3. Colleagues, I am grateful to announce that I have moved to Pinboard. 7548 links going back to May 2005 (that Faculty Academy session). The first few are podcast links. And we still have podcasts. So that’s another good thing (so far).

    The deluxe Pinboard subscription is 25.00 a year. I would gladly pay such fees to services so they could stay small, stay lightweight, and stay on. I hoped Flickr would be that way when I bought a Pro account.

    Verizon, Yahoo, Oath, ugh.

  4. “A monster post, some ranting on companies like Storify who offer free services that leverage our effort to get worth enough to get sold – when they do they just yank our content, an approach for local archiving your storify dying content, a new home spun tool for extracting all embeddable content links and how to use it to create your own archives in WordPress.”

    I had to read that paragraph twice to understand it. I’m interested in this story, but I’ll find someone who writes clearly, and let them tell me about it.

  5. A couple of things:
    1. Once again, you speak truth re: owning your stuff.
    2. The obvious advantage and perhaps the only feature here is convenience. They’re doing the lifting(not really even that heavy but still) for you. That is still valuable to lazy folks like me.
    3. Could you also save a step here but using IFTTT to capture all the hashtags/tweets and then doing a cut and paste into WP? If you say yes, I’ll feel good about myself.
    4. You spelled wacksack wrong but commend the effort.

    1. Convenience is always a double edged thingamabob, and it’s fine if you have some sense of the cost. My worry is people just take the offer w/o understanding the costs. Facebook looms as a behemoth of convenience and denial of cost.

      I am all for avoiding doing grunt work. And believe me, Storify was more than just saving time- the way it allowed you to pick and choose from multiple sources was clever. I am still gobsmacked (that’s for the British readers) that nearly every post and news site I read about the reaction to Storify’s shuttering talked about it only as far as being a means to curate tweets. They missed the bus and the boat.

      I’m not all that broke up about it, because mostly my Storify things were not terribly crucial. A few will leave gaps in some other sites like ds106, and I will grab my links for those sites before they the lights go out in May.

      I’m not sure what you mean by IFTTT saving steps. If it can grab a series of tweet links, yeah sure. There is no one right way to do it. But that again (a) only works for tweets; and (b) if all you are doing is grabbing every tweet for a hash tag. Again, my uses of Storify was not a twitter archiver, but for selective curation. Not sure how you IFTTT that.

      I’m working in my wacksack spelling! I recently got bupkis spelling right after someone else chastised me.

      Just keep blogging.

  6. Hi

    thanks for this great article and the tool you provided. The only Problem is, that there are a lot of articles (at least in my storify-Archive) which in wordpress did not bring more than just a link. No thumb, no exerpt.

    So besides of the question if it is really worth all the effort to Transfer more than 200 stories into WordPress, my issue is how to proceed after May.

    Are there any usefull alternatives for Storify to use where a Story can be embedded in wordpress Posts?

    So far I did not find a handy alternative.

    Suggestions are more than welcome

    Cheers from Germany
    Hansjörg

    1. Hi Hansjörg

      Yes, that is a gap. I discovered it myself. The WordPress oEmbed feature cannot do this with plain URLs. I am guessing storify was using Embed.ly as a service to enable this (it’s not free).

      WordPress can autoembed a page or post from another WordPress URL if you add /embed on to the end of the URL, like for this post, you would use http://cogdogblog.com/2017/12/storify-bites-the-dust/embed but that’s not a solution for what you describe.

      Personally I am not interested in other 3rd party services. Some people are suggesting this service https://wakelet.com/storify

      Before I was tinkering with this approach, I was looking at just saving the plain HTML Storify exports, and might consider doing an iframe on URLs I manage.

      1. Hi Alan

        the best way would be a Plugin with the ability to Import any Content directly into WordPress. Unfortunatly so far I could not find any.

        Cheers from Germany
        Hansjörg

Leave a Reply

Your email address will not be published. Required fields are marked *