Mastering Conversational AI: How to Construct a Knowledge-Infused Q&A ChatBot

In this article, we will explore how embedding can be used to build a knowledge base for your company. Imagine having an AI-powered repository of best practices and domain-specific knowledge at your fingertips. This knowledge base can empower your organization's day-to-day decision-making processes. While fine-tuning large language models can achieve certain outcomes, embedding provides a unique approach to harnessing your company's specific knowledge.

Understanding the Difference

Fine-tuning a large language model is ideal when you want to mold the behavior of the model for specific tasks, like making it talk like a particular person. However, it may not be the best option for retrieving specific data or knowledge. This is where knowledge base embedding comes into play. It excels at giving a large language model specific domain knowledge, such as your company's standard operating procedures or proprietary data.

What Is Embedding?

Embedding is a technique used to represent data in vector form, showcasing the relative distances between data points. Think of it as a way to measure how similar or dissimilar data points are across multiple dimensions. While it may seem like data points are mapped in just two dimensions, in reality, they can be mapped in hundreds or even thousands of dimensions. Embedding models, such as OpenAI's embeddings, are pre-trained to understand the relationships between different data points, making them powerful tools for various types of data, including text, images, audio, and videos.

Vector Databases

Vector databases, like Pinecone and Chroma, are designed for storing and efficiently retrieving vector data. They play a crucial role in making similarity searches possible. When a user poses a question, the embedding model vectorizes it, enabling vector databases to identify relevant information within your company's documents based on the user's inquiry.

Core Business Use Cases

Knowledge base embedding can be a game-changer for business automation and workflow. Traditionally, company knowledge often resides in the minds of a few individuals. Articulating best practices can be challenging, and when these key employees leave, their knowledge leaves with them. Creating lengthy documents is a common but ineffective solution, as they quickly become outdated and lack quality control.

With knowledge base embedding, you can transform emails and customer interactions into embeddings. When a customer support agent receives a query or complaint, the system can perform a similarity search within the knowledge base to find past occurrences and the best responses. This approach bridges the gap between top-performing employees and junior staff, enabling you to create a knowledge base and have the large language model learn and replicate the behavior.

In the following example (with source code at the bottom). I will present a use case based on existing knowledge, how we can embed them with AI, and then ask some domain specific questions.

Steps to Build Your Knowledge Base

Step 1: Prep Knowledge Data

Begin by preparing your knowledge data. This could involve extracting relevant data from emails, documents, or other sources. This data will serve as the foundation for your knowledge base.

Step 2: Create Embeddings

Use an embedding model, such as OpenAI's text embedding model, to convert your data into vector representations. This step will enable you to measure the similarity between different data points effectively.

Step 3: Similarity Search

Develop a function for similarity search. When a user poses a question, use the vectorized query to search for the most relevant documents or data points within your knowledge base.

Step 4: Retrieval Augmented Generation (RAG)

Set up a large language model, like GPT-3.5, to generate responses based on the retrieved knowledge. Feed the search results into the model, enabling it to generate responses that mimic the style and content of your company's best practices.

Step 5: Deploy

You can deploy your knowledge base and AI components as a web application using platforms like Streamlit or no-code alternatives such as Relevance AI. This makes it accessible to your team members, allowing them to generate responses based on the knowledge base.

Sharing Your Knowledge Base

Once your knowledge base is ready, you can share it with your team members. Streamlit is a convenient tool for creating a user-friendly interface to interact with your knowledge base. You can also deploy your application online or embed it within your company's website.

No-code platforms like Relevance AI offer an easier way to set up knowledge embedding workflows and can be integrated into your existing tools and processes. These platforms often provide APIs for seamless integration into various workflow builders.

By leveraging knowledge base embedding, your organization can automate responses, streamline workflows, and ensure that valuable institutional knowledge is preserved and shared effectively.

I hope you find this exploration of knowledge base embedding helpful for your business's automation and workflow needs. Feel free to reach out if you have any questions or need further clarification.

Demo Python Code

• Install dependencies

conda create --name faq python=3.11
pip install streamlit fastapi openai python-dotenv pydantic langchain faiss-cpu tiktoken

• Prepare FAQ_ABC Motors.csv

This FAQ set of ABC Motors is generated by GPT-4. It has nothing to do with any company.

• faq.py

import streamlit as st
from langchain.document_loaders.csv_loader import CSVLoader
from langchain.vectorstores import FAISS
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.prompts import PromptTemplate
from langchain.chat_models import ChatOpenAI
from langchain.chains import LLMChain
from dotenv import load_dotenv

# Load environment variables from a .env file if it exists
load_dotenv()

# 1. Vectorize the Q&A CSV data

# Initialize a CSVLoader to load data from a CSV file
loader = CSVLoader(file_path="FAQ_ABC Motors.csv")

# Load documents (Q&A) from the CSV file
documents = loader.load()

# Initialize OpenAIEmbeddings to obtain embeddings for the documents
embeddings = OpenAIEmbeddings()

# Create a FAISS vector store and index documents with their embeddings
db = FAISS.from_documents(documents, embeddings)

# 2. Function for similarity search

# Define a function to retrieve similar responses to a given query
def retrieve_info(query):
    # Perform a similarity search using the FAISS index, retrieve top 3 similar responses
    similar_response = db.similarity_search(query, k=3)

    # Extract the page content from the similar responses
    page_contents_array = [doc.page_content for doc in similar_response]

    # Return the array of similar page contents
    return page_contents_array

# 3. Setup LLMChain & prompts

# Initialize a ChatOpenAI model with a specific configuration
llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo")

# Define a template for prompts, which includes instructions for the model
template = """
You are a professional Q&A representative of ABC Motors. 
I will share a prospect's question with you, and you will give me the best answer that I should send to this prospect based on past best practices.
You will follow all of the rules below:

1. Response should be very similar or even identical to the past best practices, in terms of length, tone of voice, logical arguments, and other details.
2. If the best practices are irrelevant, then try to mimic the style of the best practices in prospect's messages.
3. Do not make up things.

Below is a question I received from the prospect:
{message}

Here is a list of best practices of how we normally answer to prospects in similar scenarios:
{best_practice}

Please write the best answer that I should send to this prospect:
"""

# Create a PromptTemplate using the template and specifying input variables
prompt = PromptTemplate(
    input_variables=["message", "best_practice"],
    template=template
)

# Initialize an LLMChain with the ChatOpenAI model and the prompt template
chain = LLMChain(llm=llm, prompt=prompt)

# 4. Retrieval augmented generation

# Define a function to generate a response using the retrieval-augmented generation approach
def generate_response(message):
    # Retrieve best practices based on the input message
    best_practice = retrieve_info(message)
    
    # Use the LLMChain to generate a response based on the message and retrieved best practices
    response = chain.run(message=message, best_practice=best_practice)
    
    return response

# 5. Build an app with Streamlit

# Define the main function for the Streamlit app
def main():
    # Set Streamlit page configuration
    st.set_page_config(
        page_title="Frequent Asked Questions", page_icon=":dog:")

    # Display a header for the FAQ app
    st.header("Frequent Asked Questions :dog:")
    
    # Create a text area for user input
    message = st.text_area("Question")

    if message:
        # Display a message indicating that the best practice answer is being generated
        st.write("Generating best practice answer...")

        # Generate a response using the provided message
        result = generate_response(message)

        # Display the generated response as an information message
        st.info(result)

# Check if the script is being run directly (not imported)
if __name__ == '__main__':
    # Call the main function to run the Streamlit app
    main()

• To run this

streamlit run faq.py

• Streamlit web GUI http://localhost:8501/. Port may be different. It should open automatically.

Demo Result

Note that it matches the FAQ table, Question 84

Copyright statement: Unless otherwise stated, all articles on this blog adopt the CC BY-NC-SA 4.0 license agreement. For non-commercial reprints and citations, please indicate the author: Henry, and original article URL. For commercial reprints, please contact the author for authorization.