5 lessons learnt from integrating AI chat discovery into knowledge portals for analyst firms

Content Catalyst has a deep understanding of analyst research firms, acquired during 20 years of working in the industry. We’ve combined this sector knowledge with AI expertise to design a faster and more immersive way of accessing analyst research.

It hasn’t been easy or cheap, and we’ve encountered many challenges along the way. Here are five lessons we’ve learnt:

1. Make your chat interface ‘familiar’

With the explosion of ChatGPT, your customers likely have some experience using AI-powered chat tools. So, now is not the time to break with design convention.

The user interface must be intuitive with clear signposting and look and work in a familiar way. Familiarity can be created by mimicking the successful structures and layouts set by popular chat tools and overlaying them with your unique brand and tone of voice.

2. Surface content within a license

The answers your AI chat engine returns must respect user licensing.

For example, if a user searches for European coffee trends but only has a license for the tea category, they need to be shown that the coffee report exists but cannot access it without upgrading their subscription.

To achieve this, your AI must have some way of knowing who users are and what level of access they have. We integrated MyCatalyst into our proprietary knowledge portal, which has flexible licensing, so the AI only returns references to sources within a user’s subscription.

3. Support the user workflow

Search is one step in a flow of actions that customers of analyst research will take to complete their task. Your AI search tool must integrate seamlessly into this flow, ensuring users receive fast, accurate, and usable answers.

What happens next? How will they use the insight and data? Perhaps they’ll want to copy and paste what they’ve discovered and share it with a colleague, create a report, turn it into a presentation, or deep dive into the source material.

When designing chat discovery, consider the next logical steps and help users complete these tasks. For instance, the next stage of development for MyCatalyst is to integrate AI with existing workflow tools within our platform, such as clippings.

4. Protect against incorrect answers

Hallucinations are incorrect answers that users may not know are wrong. Due to the nature of analyst research, your AI chat tool must deliver accurate answers.

While it’s impossible to protect entirely against hallucinations, guard rails, like beta testing and a phased roll-out, must be in place to mitigate risk.

It’s also essential to consider the weighting of results between recency and relevancy. For instance, the AI may conclude that a 2022 coffee trends report is the most relevant, but the data may be too out-of-date to be helpful to end users.

5. Understand the commercial costs of Artificial Intelligence

There are two costs to think about when considering integrating AI chat discovery into your knowledge portal.

The first is the substantial costs of researching, developing, and training the AI. You can expect to spend upwards of £100k.
The second is to forecast how much it will cost you to run each month. AI services like ChatGPT don’t make this easy since they base price on the number of conversations. A conversation is one initial question with four follow-up questions, each becoming more expensive with the increase in processing.

Due to the costs and unknowns associated with building in-house, we recommend analyst firms invest in ready-made solutions like MyCatalyst. For a smaller investment, you can quickly release a beta tool to a small group of users.

A trial has the dual benefit of providing stakeholders with a proof of concept while ensuring that AI will create value for your business and provide a return on investment.