When we re-engineered the site late last year, we decided to implement a form of personalization by offering people choices about what they receive in the email blasts. We quickly discovered that people are not interested in one thing but many things. But to our horror, we also discovered that there were no obvious patterns to what people wanted to see. They might choose 2,3,4,5 or more topics and despite our best efforts, it was clear that we could not sensibly manage random patterns of interest.
It also occurred to us that people's interests might change over time so while we had lists of topic interest, we had no good way to determine whether we are delivering the content people want without managing at least 18 lists - that was the number of interest categories we created.
Apart from a small number of subject matter experts who focus on a narrow niche for long periods, people's interests change more frequently than might seem obvious. That makes sense when you think about how life events influence the things upon which you spend your time, at different points along life's winding path. Reflecting on my changing interests, it was clear to me that we needed to find a way to respond to that dynamic.
That thought process got us wondering if it is possible to send subscribers content based upon their activity at points in time. The short answer is 'yes.' The long answer is 'only if' certain conditions are fulfilled.
What has changed?
- Subscribers can choose from daily, weekly or monthly updates. Is that good or what? It is one more way of putting the user in charge of their diginomica experience. Existing readers can update their preferences using the subscribe functionality. It's a bit clunky today because it tells you that you need to check your inbox as if you're signing up for the first time. That's not the case and we have a bit of a tweak to apply so it doesn't confuse folks and we suck less again. But trust me, it DOES update your preferences in the background.
- We have ditched the category selections for the purpose of the email blasts. You can still visit your favorite cornerstone topic page via the menu above on a day to day basis. That hasn't changed. But before some of you yell 'foul,' read on to see how you benefit from the changes.
- Say for example that this week you want to know about HR topics and skim through a bunch of articles on that. The engine we've deployed should provide more of that content for you on the next email blast. However, when deploying any technology of this kind, the results are dependent upon what the engine learns about your behavior.
- In our case, the engine iterates over time and effectively creates a reading profile which it can then use to automatically deliver content in which you might be interested. In turn, the engine assembles a templated email of 8 pieces of content with the headlines and intro words related to that content.
- For those who have subscribed but haven't visited the site enough times for the engine to develop a preference understanding, the engine spits out the most popular content from the last period and uses that as the basis for the templated email. The same applies to those who only visit the site at random intervals to read about random topics.
- We still get the option to edit the email blast content. For example there might be a piece we are reasonably certain most people would be interested in, but which hasn't hit the most popular group. We can manually insert that if we wish.
- In similar fashion, we can exclude content that while popular, is not wholly relevant to the intended reader. That has only happened twice and was the result of consumer pickup of a story that sent the content wildly viral.
Will it blend?
This is a stepping stone. We know that as the system has taken hold during our test phase, the content open rates have improved and are now 50% better than those we were achieving on average before this method was introduced. The article open rates are 6x higher overall. We expect those numbers to improve as time goes on.
During dev and test, we endeavored to discover the right day/time for delivery. It was a classic A/B test. When we apply A/B testing to content we select manually for an email blast, we see specific day/time combinations where content works best. That didn't happen in the new system.
Instead we saw similar results regardless of the delivery day/time. We're still mulling over what that means but suspect that it relates to the value the personalized content delivers back to the user experience. If we are correct then it means that personalization as we're currently iterating that experience is absolutely the way to go for engaged readers.
All of this is encouraging because we know that both the average open and click rates are higher than industry averages. Much higher. It also means that we can redistribute about 20-25% of the overall content created and get an uplift in engaged readership at the same time.
We believe this is the way to provide the best compromise for those readers who don't necessarily want to consume content each time there is an RSS ping or Twitter promo, but want to catch up on the diginomica analysis of events in which they're interested at a cadence that suits them.
Are we done?
That's pretty much it as of today. It's taken us four months to get here with a roadmap and dev rollout that has exposed many issues about how services need to interlock. Phil Wainewright has talked about this in the past but we continue to see improvement in the way cloud based services can be assembled into new 'things.'
We have back end 'features' (aka bugs) to squish (solve) but as we stand today, the system is working as intended for users and producing results which we believe are additive to the quality of the reader experience. That meets our initial design criteria.
We're not aggressively pushing these new features through pop ups each time a new visitor comes to the site. We think that is a very poor way to demand attention. Instead, we believe diginomica content should stand on its own merit, and if people wish to get more of the same, then we have the sign up mechanisms in place to ensure they get what they need.
What's next? That's the subject of more head scratching about the user experience and analysis of the data we're seeing. In an ideal world, I'd like to think that we could add personalization into the core CMS engine. That would mean we're delivering reader specific content live on the site when subscribed readers visit. Having a daily digest at your fingertips (if that's what you want) is a first step in that process but I think we're some way off getting to that state of grace.