Fetching Stuff from Mastodon (simplest brute force approach)

The clock is ticking for me to modify the WordPress theme that powers the DS106 Daily Create given that the slippery slope timeline for free, unfettered access to the Twitter API will sooner? later? tomorrow? hit the bottom.

Just to be clear, nothing here stop the DS106 Daily Create site from publishing it’s 4050+ day consecutive streak of daily challenges. Having them appear each day in Twitter might still work, depending if the Dark Lord of Twitter’s foulings affect the ability to tweet via an IFTTT recipe.

What it likely will mean is the website will no longer be able to get replies or mentions of replies in twitter to add to the site as a response. Since 2014 this has allowed responses to not only be added to the associated daily post (see that example via tweets including @ds06dc and the matching daily hashtag, there #tdc4057), but also provide a single user profile of lifetime activity (see an exemplary example).

Via Mastodon?

It’s obvious this can be continued by providing the same capability from Mastodon. The first steps in the process was started in late November 2022 setting up the posting to a new account via again IFTTT spawned from the RSS feed of the Daily Create site (you know, that dead technology no one uses).

Then I laid out my todo list, now indicating completion with a ?

1. Create a DS106 Daily Create Mastodon account ? (see/follow @tdc@social.ds106.us)
2. Get the Daily Create WordPress site to auto publish new posts to a Mastodon Account ? (done since Nov 25, 2022 using IFTTT, see blog post)
3. Figure out how to get replies to that account via Mastodon API
4. How to parse the replies to find hashtags
5. Add each reply as is done in twitter, to the WordPress content type that houses them
6. Figure out how to embed the Toots from a URL ? (autoembed, working here in this blog, code available, see blog post)
7. Other stuff I am forgetting

Maybe the progress is not so bad. But #2 vexed me for too long.

Finally, API Success!

There’s plenty of good documentation for the Mastodon API, of which I have a rudimentary understanding. To work in the WordPress site I need some code that will let me site make regular calls to the API to find mentions of our account, one of the types of Mastodon notifications.

The funny thing is that since 2014, for getting responses from the Twitter API this theme made use of the oAuth Twitter Feed for Developers WordPress plugin for which I made a slight tweak to return mentions rather than a timeline. It’s worked flawlessly. I just needed to pass it the API keys and it provided functions for fetching tweets.

I had hopes for Mastodon, but ran into nothing but dead ends with the listed libraries for PHP. Many of them require running through composer (which for too long a story I cannot get running on my old laptop I use for development) and even the end-around solution I found lobbed a paywall in front of me.

Plus many of them had more complexity than needed. I don’t need to run through an authentication loop since I can easily generate a development token like I needed for IFTTT.

I stalled.

Ask A Guru

I finally did what I should have done earlier- ask for help. I reached out to Pat Lockley because I thought I read somewhere he was messing around with the Mastodon API (or just because he is smart). It’s worth reading Pat’s Daily Dive posts because it’s more rewarding than social media scrolling (I would suggest doing what I did and subscribe to his RSS feed but there are apparently only 4 people left who do this).

He not only provided me the answer (11 lines of code doing requests via cURL) but believe I got sub-daily-dived.

It was enough to get me going; I was able to get my accounts notifications coming in, and I could conditionally test to process ones for mentions. A few more bouts and I was able to rig the request to only return mentions.

Fetching but Not Yet Built In

The theme pulls data from each valid response (a tweet containing the host account @ds106dc and a hashtag in the form of #tdc****) and creates a custom content type for each one (thus making an archive):

Thus what is needed to parse from the API is:

• username of tweet author
• ID for the status
• URL for the tweet
• content of the tweet
• hashtags in the tweet
• date posted (left out from diagram above, but it becomes the publish date)

These are all available in the Mastodon API, so now I have a means to check for new posts, they can be processed almost the same way. I imagine I might add a new metadata field to identify the response as a tweet, toot (or something else that might come later). Any with null values will be considered tweets.

I have a simple PHP test page running generating as called the output like below:

So, I would process only ones containing a tdc****value in the array of tags (Kevin’s post at #3 worried be but his is just a reply to another post, not dailycreate response).

Da Code

I will post a copy of my working code if you want to play. The necessary step is getting your access token for the account you want to be checking. Just go to Edit Profile -> Development then click the New Application button. The title can be whatever you want, web uri optional. The key thing is deselecting all the options for scopes; the only one needed is read:notifications. This gives the app you are creating very specific capabilities, just to be safe.

Once saved you can inspect; the string you will need to save for use is labeled Your access token

I set up my demo so the account info including this access token is declared at the top. Certainly fiddle with the output.

You can download a gist with the full php- edit with your own info, slip online somewhere, and call the URL.

It’s basic, but it works. I can add checks for items 3 and 4 on my list.

Since the Daily Create is set up to request once and hour the API calls should not stress or load your instance. I will look for ways to reach back in time. Maybe.

What’s Next

There is a heap still to do to add more options and logic o the theme (allow use of both or just one service, though if twitter goes down the Pay For API I am not sure who would. Some mods are needed to change the instructions, and other bits for parsing username into URLS.

I expect our frequent participants will want me to somehow resolve combine their responses from both twitter and Mastodon… sorry but the odds of that are massive!

This is maybe too much explanation, but I always finds it helps in the process to write it outloud.

Slowly, it’s on its way! The DS106 Daily Create will go on and soon be fetching from Mastodon.

---

Featured Image: Skinna the Icelandic sheepdog I knew back in 2008… she was a master fetcher!