The path to self-improving products

I've been thinking about the impact of artificial intelligence on product building in recent days. This has been prompted by a few things:

YouTube

As I wrote about most recently here, I'm experimenting with building a YouTube channel. For now it's just an experiment, but I've already learned a lot from it; about storytelling, experimentation, and about product too (you must deliver the 'aha' moment as early as possible to have the largest impact on your retention metrics).

With YouTube Shorts in particular, it seems to send your video out to the Shorts Feed initially to a very small number of people (10-100), and based on their interactions it determines whether it wants to push it beyond that. So far, the majority of my videos have stopped at this point, but I've had one or two that have gone beyond it, and it almost always seems to be within the same timespan since the upload. Why is this relevant or interesting? Because YouTube is applying an algorithm (most likely to include some machine learning) to determine whether it wants to push or limit a video's reach, and in doing so it is improving the product experience continuously to ensure the right content is being shown to users.

Algorithmic Content Recommendation

Expanding on this point, we've seen the same thing with companies that rely heavily on artificial intelligence to curate their product experience like Netflix, and TikTok. Airbnb are known to do the same for listings.

Where does this go from here?

I believe this is only the tip of the iceberg. With the rise of large language models (LLMs), we're going to see an increasingly rapid feedback loop, that often includes no humans at all. Rather than just impacting the content that is prioritised within the product experience, it can shape anything: layouts, user flows, feature visibility and more. One way to think of it is an ever-expanding A/B test that is running, rather than with a random sample, in an intelligent way.

Here are some examples that come to mind:

• Analytics -> Product Iteration: Currently most product teams take into account the analytics generated from user behaviour within their product, analyse it and then make product decisions based on it. These decisions are then fed into the product development process, with decisions being needed about the changes to the product to address and improve these metrics, and then the design and development work taking place to push them to actual users. What if artificial intelligence were always monitoring analytics, and taking direct action based on poor performing parts of a flow? For instance, it might notice that a certain step within an onboarding flow is suffering from a particularly high drop-off rate, and instantly submit a PR to the code repository to improve this, with changes to the copy, layout and design. Because it has access to the resulting analytics, it can intelligently understand whether the change was a positive one, and continue the process until the drop-off rate is improved. Multiply this process across a product, and your speed of iteration becomes exponentially faster than it is today.
• Self-Authoring Blogs: Similarly, the process of building a blog as a distribution channel for a company is a slow one. It can involve keyword research, writing posts, looking at the resulting search impressions and then taking action to improve search position and keyword coverage. It is for this reason that startups often retain SEO/growth agencies to handle the complexity. But what if LLMs wrote the initial seed articles, monitored the resulting SEO information (such as that available from the Google Search Console), and then made edits to the existing articles and wrote new ones to address the gaps within the keyword coverage. As the company moved into new product areas, you could direct the model to write on those keywords, and suddenly you have a self-authoring blog to drive growth to your platform.
  • The obvious fall-out from this is that once it becomes commonplace, the value of SEO will drop to zero, and search will have to evolve. As it already is.