Last week I spent three days attempting to create a usable aggregated RSS feed for our daily newsletter. It is a tale of Sod's Law in full pomp. First some background.
We know that people like to consume information in a variety of ways and that choices are dependent upon many variables. For example, I posed this question on Twitter last month and got these responses:
When running a similar poll on LinkedIn I got this result:
I'm not foolish enough to think that a relatively low level of engagement provides a definitive trend but the in results across these two networks was enough for me to think and rethink what we should be prioritizing. In that same vein, one of the elements we know people find interesting is Jon Reed's 'best of the inter webs' section in his weekly Hits & Misses. I want to incorporate something similar in the digiDaily newsletter. Why?
John Dunne's 'no man is an island' springs to mind. We are all influenced by a variety of inputs and more often than not, some of the most interesting material comes from serendipitous discovery at the edge. From time to time, people in our networks alert us to something they think might be of interest. In turn, we think it is important to demonstrate that there are plenty of incredibly interesting ideas under test and which may have broader application. To us, throwing those pebbles onto the pond of life is important. After all, you can read all the stories you want in the WSJ, FT or wherever your favorite publications are located.
In the 'good old days' of the mid 2000's much of that discovery occurred through blog links which were freely shared. Anyone remember the concept of the blogroll and good old RSS? Those formed the foundation blocks through which curious people discovered fresh thinking. Today? Not so much. Instead, we live in a world that's divided between content aggregators and those who choose to wall off content through some sort of subscription mechanism. The rise of Substack, Medium and others are testimony to that approach. It is still possible to discover content based on your interests but it is much harder to automatically share in the context of a newsletter.
When I'm in the discovery phase, I need RSS feeds from which I can select content. To that extent there are any number of RSS feed readers from which to choose. But - and here comes the kicker - those same RSS readers have not evolved much beyond their original purpose of providing a personal feed. There are exceptions. Feedly for instance allows firms to create teams that share content among themselves. The idea is that teams of marketers find or work on topics of interest that are eventually used or shared as part of a campaign. I get that. But what about those, like us, who simply want to share what we've found because we think it's interesting?
This is where life starts to get tricky. Zapier claims to provide a simple mechanism through which you can connect thousands of apps such that the information contained in one app can be consumed by another. It does this through a simple, no code interface that is sort of easy to understand. But - it is entirely dependent upon the data that is exposed by or can be consumed by the connecting applications. In the case of RSS, this ought to be straightforward but it turns out this is not the case. Or at least not as far as I can determine.
We already know from our experience of handling partner RSS feeds that getting feeds right requires a degree of skill and precision. But shouldn't it be easy for a firm like Zapier to simply 'make' a reliable RSS feed for consumption elsewhere? Apparently not. Over the course of the three days I spent in RSS hell, it turns out that the feed it produces from Feedly is at best hit and miss - no pun intended Jon. Sometimes it would be OK, other times it would duplicate content, still other times it would not include the content links and then send people back to Zapier. All in all, it was an unpredictable mess regardless of what I tried to do.
I don't want to connect Feedly directly to my email newsletter system because I want to provide context for the content being exposed via RSS. This means I want to show the title, a short copied excerpt that we control and the source. That way, people who are looking at this type of content have a better idea about whether it is content they can trust. Over time, you can imagine this being refined into topic areas where people have specific interests. But as outlined above, this proved unreliable. What's more, Feedly has the concept of annotation and shared notes but as noted, these are really for internal consumption. Or, if they're for external consumption then I'm not clear how this would work in the context of newsletters we're already developing through a dedicated system.
This is where Jon Reed came to the rescue. He's been firing out his JonERPnewsfeed for years. You can consume it on Twitter or by email. He links Newsblur to Pinboard which in turn creates a daily newsletter of what he's reading. What I want though is a team facility so that core team people can contribute to the overall third party content discovery we'd like to expose. Hence, I prefer that we end up using Feedly since it is inherently multi-user. Feedly is easy to understand and comes with a great UI. Newsblur on the other hand is ummm...clunky...if you're not used to it. Pinboard is another 'old school' solution but here's the important part - I can easily edit the content Pinboard ingests and then push that out to my email newsletter system which can selectively publish RSS feed items. That's because Pinboard provides its own, clean RSS feed where I get to control the content that's exposed. This is useful because while excerpts that are written by content authors are useful, they are rarely great. What I want therefore is to pull out the nugget that makes a particular piece of content interesting.
This is where we're at. We have Feedly connected to Pinboard which in turn generates an RSS feed that our newsletter system consumes and publishes. I can drop items onto Pinboard via the 'What we're reading' folder in Feedly so that Pinboard only picks up the stuff in which we're interested. In turn I can pull quotes from the original content and drop those into feed items. I can delete items if, on consideration, they're not as interesting as we first thought. And finally, I can consume that in the email newsletter application for general distribution. It's not as complicated as it sounds and we have achieved a level of automation that is probably about as it good as it gets. We still have to read the stuff we think is best and pull out the nuggets but that's the whole point anyway.
So far, it seems to work well. There are of course a few more wrinkles such as ensuring the Pinboard feed is up to date, that the formatting is as we expect and that we're consistent in our approach to the excerpted content. But I think we've finally got there. No doubt some of our most vocal fans will tell us but in the meantime I'm happy that we can continue to enhance our readers' experience by providing information they'd otherwise have to hunt down for themselves.
The sadness is, as some have observed, that RSS is not getting the development love it deserves. It is truly a powerful tool for content creators and consumers that needs attention.