Creative AI in business: how to adapt ChatGPT (and similar) to my customer's needs

August 14, 2024

Generative Artificial Intelligence (GenAI) has gone from being of purely academic interest in the last year to making the front pages of world news, thanks to the democratization of tools like ChatGPT or Stable Diffusion, capable of reaching the general public with free use and a very simple interface.

However, from a business point of view, is this wave of attention pure hype, or are we in the early stages of a major revolution? Is GenAI capable of generating new and innovative use cases?

The LLM (Large Language Models) era

The GenAI is a branch of AI focused on creating new content. We can consider IAG as a new stage of AI: with traditional statistics you can only have descriptive and diagnostic analysis.

  • We can move on to predictive and prescriptive analytics with AI (especially with machine learning techniques), which is able to foresee patterns or situations and offer recommendations or alternatives to deal with them.
  • By using GenAI, we have creative analytics, capable not only of studying existing data but also of generating new information.

Although GenAI is applied to all forms of human creativity (text, audio, image, video...), possibly its best-known aspect are the so-called Large Language Models (LLM), especially OpenAI's ChatGPT, which has the honor of being the fastest growing App in history. Rather than using machines' language, we can speak our own natural language with them.

'Traditional' AI (in this field anything more than 5 years old is already traditional) undoubtedly meant a change of mentality for companies, equivalent to the democratization of computing in the 80s and the internet in the 90s. This digital transformation makes it possible to optimize processes to make them more efficient and secure, and to make data-driven decisions, to mention two of the main applications.

Do we have equivalent use cases with GenAI?

Some of them, proposed in these early stages of GenAI, include:

  • Internal searches for documentation: having large volumes of unstructured information (text), specific information on concepts, strategies or doubts can be consulted in natural language. Until now, we did not have much more than Ctrl+F to search for exact correspondences, which is not very efficient, very error-prone, and only provides fragmentary information.
  • Chatbots: with the precision and versatility achieved by the latest LLMs, chatbots are able to respond much more fully to user questions. This allows the vast majority of problems that customers may have to be solved with great agility, requiring human intervention only in the most complex ones. It is even possible to perform feelings analysis on past responses, to understand which strategies and solutions have been the most satisfactory.
  • Synthetic data generation: from a private dataset, we can swell its volume to provide a larger sample. This is especially useful when obtaining real data requires a lot of time or resources. With IAG, it is not necessary to have advanced knowledge of statistical techniques to do this.
  • Proposal writing: using existing documents, we can generate new value proposals aligned with the company's strategy, according to the type of client, team, type of use case, deadlines, etc.
  • Executive summaries: also from documents, or voice transcriptions (another use of GAI), we can summarize long and complex texts, often of a technical or legal nature, even adapting the style of the summary to our needs. Thus, if we do not have time to attend an important committee, to read 120 pages of a proposal or to understand a new European legislation, the GAI can summarize the most relevant points in a clear and concise way.
  • Personalized marketing: 'Traditional' AI enables customer profiling and segmentation, which facilitates the design of personalized campaigns. Using GAI, marketing can be created on an individual level, so that no two interactions with each customer are the same, and always in line with the company's existing values and campaign style. This opens the door to creating content on a scale that would be impossible to create manually, as with automatically generated YouTube captions.

Retraining or adapting... or both

It all sounds great, but how can we have an LLM for our company? The most obvious option is brute force: we train one of our own.

Unfortunately, it is no coincidence that the best LLMs come from the so-called hyperscalers: OpenAI, Google, Meta, Amazon... training one of these models requires an exorbitant amount of data, time, and resources.

◾ As an example, the training of GPT-4 cost about 100 million dollars, a figure within the reach of very few pockets.

On the other hand, as ChatGPT-3 reminds us every time we ask it about something current, its training was done with data up to September 2021, so it does not know anything later. However, its successor GPT-4 did incorporate conversations with GPT-3 as part of the training.

This meant that, if a user in one company had revealed secrets to it in the chat of the previous version, another user in GPT-4 could access that information simply by asking for it. As a result of these incidents, many companies have banned its use to prevent leaks of confidential data.

Therefore, if we cannot train our own model natively, we have to adopt other strategies to make the LLM work with our data and company casuistry. There are two strategies, fine tuning, and retrieval-augmented generation (RAG). There is no one better than the other, but it depends on many factors.

  • Fine tuning: as its name suggests, the idea is to refine the model, but keeping its base. That is, take an existing LLM like ChatGPT, and train it by incorporating all the data from my company. Whilst it can be somewhat costly, we are talking about orders of magnitude less than the millions of dollars involved in doing it natively, as it is a small training set compared to all the information that the base model has gobbled up.
  • RAG: In this case, all we do is create a database of information relevant to the customer, so that they can ask the LLM questions about it. When faced with a query, the LLM will search through all these documents for pieces of information that seem relevant to what has been asked, will make a ranking of the most similar ones, and will generate an answer from them. Its main advantage is that it will not only tell us the answer, but it can also indicate which documents and which parts are used to create it, so that the information can be traced and verified.

Which of the two is more suitable for my use case or customer?

As we have said, it depends. We must take into account the volume of data available, for example. If our customer is a small company and is only going to have a few hundred or a few thousand documents, it is difficult to do a satisfactory fine tuning (we will probably have overfitting), and the RAG approach will be much more appropriate.

Privacy is also fundamental

If we need to restrict the information that employees at different levels or departments can access, we will have to make a RAG, in which different documents are available depending on the type of access. The fine-tuning approach is not suitable, as there would be a potential data leakage similar to the one discussed in GPT.

How versatile and interpretable should our model be?

This point is crucial, because choosing the wrong strategy depending on the use case can ruin it. If we want our LLM to summarize a document for us, for instance, or to make a mix of several documents to explain something to us, we want it to be literal and traceable.

LLMs are very prone to hallucinate, especially when we ask them something they don't know. Thus, if we use a fine tuning strategy and ask them to explain something that is not in any document, they may pull a fast one and start making things up.

However, if we have a chatbot for customer service, we need it to be very flexible and versatile, because it must deal with many different casuistries. In this case, fine tuning would be much more suitable than RAG.

The refresh rate of our data is also important

If we are mainly interested in taking into account past data, knowing that the rate of new data will be very low, it may be interesting to adopt a fine tuning, but if we want to incorporate information with a high frequency (hours, days), it is much better to go for a RAG.

The problem is that fine tuning "freezes" the incoming data, so that every time we would like to incorporate new information we would have to re-train it, being very inefficient (and expensive).

A compromise between the two approaches

There is a compromise between the two approaches, consisting of fine tuning to incorporate a large corpus of past information from the company or customer, and using a RAG on top of this more versatile and adapted model, to consider recent data.

◾ Following the chatbot example, this would allow us to have a virtual assistant capable of dealing with a range of situations based on the history of resolved incidents, and at the same time have updated information on the status of open incidents.

Once we have all these factors in mind, we can choose one of the strategies, or combine them, in order to create a customized ChatGPT for different customers, thus generating an attractive value proposition with different use cases.

References:

Generative AI as part of business strategy and leadership
Cyber Security
AI of Things
Generative AI as part of business strategy and leadership
September 20, 2023