How Phrasee’s new content engine incorporates LLMs, but maintains control

Generative AI is all the rage now, but if you're in marketing, you know there have been martech vendors around for a while that do what ChatGPT does. I caught up with one of these vendors - Phrasee.

Matt Simmonds, Chief Product and Technology Officer for Phrasee, shared his insights on generative AI, Large Language Models (LLMs), and Phrasee's new Content Engine, which includes integration with LLMs.

Incorporating LLMs

It's been an interesting six months for Phrasee as they learned to navigate the discussion with potential customers of generative AI and large language models. You see, Phrasee is eight years old, and they've been doing generative AI for that long - it just wasn't called that. And so discussions start with how these things fit with Phrasee? Simmonds said it has opened the world to what Phrasee does, but it has also potentially created‌ hundreds of competitors. He believes Phrasee brings unique differentiators to the table, though.

Simmonds said everyone wants to use generative AI now, but many don't know where to start (great stats from Salesforce to back this up). That's because there is so much more to generative AI than being great at prompt engineering. 

Phrasee has spent the last two years working on updating its tech, which has involved designing a different approach to content generation that incorporates LLMs, but considers control and performance. LLMs are unconstrained, Simmonds said; there is no control. It's the missing piece to many solutions that build on these large language models.

Doubling down on the original pitch with a difference

Phrasee's original pitch was all about generating and optimizing content at scale. With their new Content Engine, they are doubling down on that, with some added capabilities. Before, the focus was on short-form content like subject lines, headlines, push, and SMS messages. Now, they add the ability to expand into more medium and long-form content like product descriptions, emails, social posts, articles, and blogs. 

But it's not just about creating content. It's ‌about creating good content that engages the audience. Because anyone can spin up the free version of ChatGPT and prompt it for content. Whether that content hits the mark is the question marketers need to know, and it would be nice to have some idea before that content goes live.

For Phrasee, that meant improving their proprietary AI. So here's how the Content Engine works. The marketer selects a content type and completes a content brief. This brief is basically built-in prompt engineering. The request is then submitted to the Content Engine, which produces content from a combination of Phrasee's content graph (their proprietary tech) and output from several LLMs. The integration of LLMs enables Phrasee to create new longer form content types. 

It then layers in performance predictions that leverage a deep learning model trained on eight years of content experiments, adds in checks for brand style and voice, and content diversity. 

Simmonds said you need to see a diverse set of content assets to drive performance. Diversity is all about creativity, saying things differently with different syntax or linguistic approaches. Getting that out of ChatGPT without many different prompts is very hard.

You also won't get just one content asset through this process. Phrasee provides a set of content assets to compare. 

The last part of the content generation process is review and approval. Like I've heard from almost every tech leader working in AI, Simmonds said we will always need a human in the loop, which is why there is a review and approval workflow. The other feature that Phrasee offers is built-in AB and multivariate testing. Before the release of Content Engine, testing was always part of the process. Now, it's optional because you don't need to AB test mid-form or long-form content. 

Hallucination is a long-form content problem

We talked about the hallucination challenge that comes with large language models. Simmonds said it's not really a problem with short-form content, but what can be challenging is the lack of understanding of intent. He said the content graph allows semantically linking topics together to improve the understanding of what's asked for in the brief. However, hallucinations will still happen for long-form content; it's impossible to stop.

But Simmonds does not see long-form content as the primary goal for marketers when it comes to generative AI. He sees it in ‌short and medium-form content like subject lines and social posts. Simmonds believes that generative AI has the power to transform true content marketing, which is always performance-based. (I'll insert here that Simmonds isn't talking about content marketing in the way many think of that term - which is blogs, whitepapers, and similar content).

On the path to true 1-1 personalization

Phrasee is also working on new personalization capabilities, not quite to market. These capabilities require this new Content Engine to be in place. It's something that Jasper Pye, VP of Product, talked about with Tom Wilson earlier this year. Simmonds said: 

The idea is that historically we've optimized for the whole audience, but we recognize that our customers, in the pursuit of personalization, are getting a smaller and smaller, and more targeted audience. And it means that optimization doesn't make sense. 

So we're building a feature which is going to do the opposite of that. So rather than having 10 headlines or subject lines that you're looking to find the best one for the audience, we would generate potentially 1000s of outcomes, and then we would serve the best outcome for an individual based on what we've tracked about that individual. 

With the new content engine in place, Phrasee is working on enabling brands to generate messages that match the profile and preferences of each customer. Essentially, they are building up a history of language and content preferences for each customer based on what they engaged with in the past. Marketers can then query Phrasee to determine what to say to a customer and in what style. 

For example, Phrasee would integrate with an ESP (email service provider), and for each email that needs to be sent to a customer (e.g., a cart abandonment email), ‌the ESP would query Phrasee on what to say and then send that content in the email. This is all fully automated, in real-time, and at scale. Exciting and scary at the same time, right?

The money isn't in the LLMs; it's what's built on top

Simmonds said that all the LLMs are fighting for dominance, but there's nothing unique about them. He said it's all about computing power. LLMs are a commodity, and they will become basically data pipes.

The companies that will win, he said, are the ones that are building something on top of them. The companies will harness the data best and provide the control, workflow, and tools that make them usable. He believes Phrasee is leading the way in this space. 

Harnessing the power of LLMs inside the Content Engine is what will help Phrasee get in front of other marketing teams in the organization. Product marketing, content marketing, and others need medium and long-from content generation capabilities that generative AI can provide. 

My take

I like Phrasee's approach to generative AI - the mix of their content graph, which has been running content experiments for years, plus the addition of LLMs. But what's even better are the additional features around performance and control that ensure content is on brand and should perform well. The fact that it generates multiple versions of a content asset is also very nice because it gives you options to test without the effort of going back and forth to create them.

I've tested generative AI tools, including ChatGPT, on all types of content, and in some instances, I've been impressed. In others, not so much. So I understand the value of diversity that Simmonds talked about. Yes, some of these tools can be trained on brand voice and style, but can they be trained to create a diverse set of content options that are still on brand and will perform well? These are the features marketers will look for after they get past the shiny new object syndrome of generative AI.