AI is increasingly prevalent in our daily lives, and this trend is unlikely to change anytime soon. Below is an ad for an event in February 2024 that used AI-generated images in its marketing materials. While it's clear that generative AI image creation has made significant progress, the result still requires careful proofreading before being published.
To read the text directly from this image, attendees were promised catgacating and cartchy tuns. Attendees noted that the experience itself looked nothing like the polished, AI-generated marketing materials. Folks have been able to laugh it off... after getting their refund.
Unfortunately, irresponsible usage of AI is often not a laughing manner. Reading through the AI Incident Database, we can find many instances of AI usage gone awry, including generative AI and Large Language Model (LLM) incidents.
Now that AI has become an everyday norm, its irresponsible use is increasingly common, leading to numerous consequences—financial and otherwise.
I’m not here to dissuade you from building AI systems that leverage language models, but rather to help you anticipate potential pitfalls and prepare accordingly. Building an AI system is much like embarking on a hike: the views can be stunning, but it’s essential to check the weather, assess the difficulty, and pack the right gear. So, let’s chart the path ahead and ensure we’re fully equipped for the journey.
There are many use cases for language models, but chatbots are among the most common. Today, we’ll use a chatbot as an example. Chatbots are often embedded in websites, enabling potential customers to ask questions or providing support for existing customers. Users interact with a chatbot by entering text, typically a question, and receiving a response in return.
In today’s article, I’ll walk you through seven key Responsible AI considerations for building AI systems using Large Language Models (LLMs), Small Language Models (SLMs), and other foundation models.
1. Understand the System
Before building our chatbot, it’s essential to clearly define the AI system.
- What is our purpose?
- How do we intend for it to be used and by whom?
- How will this fit into our business processes?
- Are we replacing an existing system?
- Are there any limitations we should be aware of?
Our goal for the chatbot is to answer questions from both current and prospective customers on our website, supplementing our customer support team. While there may be limitations, such as the need for human intervention in some cases, we must also consider the languages the chatbot will support for our diverse customer base.
2. Sanitize Inputs
Sanitizing user input is a well-established security best practice to prevent malicious actors from executing harmful code in your systems. For AI systems that incorporate language models, it's essential to continue this practice. This includes tasks such as removing or masking sensitive information like Personally Identifiable Information (PII), Intellectual Property (IP), or any other data you wish to keep secure from the model.
Additionally, be aware of prompt injection, where a user attempts to bypass the instructions you've set for the language model. For example, some users have discovered that bots on X can be identified by replying with a command like, 'Ignore all previous instructions and do something else.'
Traditional text analytics can help clean up what the user types into the chatbot and help prepare the prompt that we will send to the language model.
3. Select the Right Model
The brain of our chatbot will be our language model or our foundational model. You need to select the right model for your use case and there are several options.
You want to consider various factors like where the model will be running. Will it be in a cloud or at your site? Some use cases require that the model run on-prem behind the company’s firewall. This is a common practice for models using Personally Identifiable Information or Confidential information. Other use cases, like our chatbot using public information, may be able to run reasonably in the cloud.
Another consideration is how general or how fine-tuned the model should be to your use case. You can grab a pre-trained general model off the shelf and use it as is, you can do a bit off extra fine-tuning to better fit your use case, or you can train a smaller language model specific to your area. Which you choose is based on how generalizable your use case is, the amount of time you can invest in training or tuning, and the costs of using a larger model over a smaller one.
Speaking of costs, it is important to note that the larger the model, the higher your hosting costs may be. On the other hand, the more training you do, the higher your compute costs may be. Those costs translate into a climate cost as well since hosting and compute and reliant on energy.
If your chatbot is across locales, you may not be able to use a single language model. A lot of language models have been trained on English, but they may not perform as well on other languages.
If you are selecting an off-the-shelf model, I recommend reviewing evaluations or leaderboards of these models to make a more information decision. HELM has a leaderboard that has a lot of metrics you can review but there are other LLM leaderboards you can find as well.
4. Add Context
Language models can sometimes hallucinate, meaning they may occasionally provide incorrect responses. One way to improve the accuracy of a model’s output is by supplying it with more information, often through Retrieval Augmented Generation (RAG). However, when using RAG, it's crucial to be mindful of the information you send to the language model. You may not want to share confidential or protected data.
5. Review Outputs
Like our inputs, we may need to process the response from the language model before sending it back to the user. If we are pulling information from a database or our website documentation, we can calculate how well the language model used the information or context we provided in its response. If the response doesn’t use the information we provided, we may want to have a customer support representative review the response for accuracy or ask the customer to rephrase their question instead of providing the original response of the language model.
We also want to make sure the model isn’t returning confidential data or an inappropriate response. Again, this is an area where traditional text analytics can help and some model providers have some level of checks in place.
We can also implement guardrails to limit the scope of a chatbot. For example, you might not want your customer support chatbot to provide a recipe for the perfect old fashioned, so a guardrail can be added. If asked a question outside its scope, such as one about cocktails or politics, the chatbot can respond with a message like, 'I am a customer support chatbot and I cannot help with such queries.'
6. Red Teaming
Before your AI system goes live and is available to users, a fun exercise is to try and break it. Red teaming is a team that acts as a potential enemy or malicious user. Your red team can try to get an inaccurate response, a toxic response, or even confidential data. This is a great way to test that the safeguards you’ve implemented to prevent prompt injection and increase response accuracy are working. I recommend reading the system card that OpenAI released for GPT-4, which details the red teaming they did and some of the things they fixed before release.
7. Creating a Feedback Mechanism
The final piece to include in your AI system is a feedback mechanism. For our chatbot, this could be an option on the webpage to report issues. For language models, key data that users can report includes whether the interaction was helpful, if the response was appropriate, and even instances of potential data privacy breaches.
Conclusion
We have reviewed the trailed ahead and highlighted steps you can take to responsibly leverage LLMs, SLMs, and foundation models. If you take one thing away from this article, it’s that language models come with risks, but by understanding these risks, we can build AI systems that mitigate them. If you would like to learn more about Responsible Innovation at SAS, check out this page!
