Recently I attended a workshop on how to build a GenAI chatbot from scratch on our local system, I will try to emulate the steps involved below, so that the readers can benefit from it too.¶

Prerequisites- VSCode application, a small LLM model, we used gemma2:2b from Ollama website, and python 3.10 installed locally on the machine¶

In [1]:

#first step - 
#we need to create a folder where we will store the files related to chatbot
#then we need to access our bash/commandprompt to access the folder by specifying the path using the cd, changedirectory command
#suppose our folder path is c\user\chatbot, then we need to copy this path and in our cmd, commandprompt type 'cd c\user\chatbot'
#when the cmd is showing path as c\user\chatbot, type 'code .'  

#this 'code .' opens the vscode app directly from the commandprompt

In [3]:

#it will open the vscode, as shown in the below screengrab - (this shows my completed project)

The architecture of the chatbot is straightforward, it will be interactive with the customers, so we need system prompt and human messages, and in the process, we need to create/feed the system, the text/data about our business/service to the system, and for the system to understand, we need to tokenize the data and store it with our chatbot database, and for the bot to create meaningful messages in reply to human interaction, we will need the earlier mentioned gemma model and for the UI interface, we used the streamlit app¶

In [5]:

#step2- click on the 3 dots as visible on the very top of vscode application and create a new terminal

make sure that you select commandprompt in terminal, it has powershell by default¶

In [6]:

#step4 creating virtual environment for the project with python 3.10

when we are command prompt, the address should be indicating that we are in the folder that we created, to create virtual enviornment following code is used -¶

py -3.10 -m venv myenv¶

to activate this virtual enviornment - we can use the above code in the pic¶

myenv\Scripts\activate¶

In [7]:

#step 5- 
#creating the text file about our business/service
##Mr A's caffe is a small family restaurant in the outskirts of the city. It serves the patrons with the loal delights and desi cuisines which are finger licking good

we will be creating 5 new files inside of our chatbot folder, so to create one, navigate to the name of the folder and click on the '+' icon to create new file, and give name and extension of the file -¶

installing the required libraries, like in our jupyter notebook project, before doing anything, we install all the libraries¶

to install these, type in the command prompt- 'pip install -r requirements.txt'¶

In [10]:

#step7, this will take some time, so please be patient, now we will need to create the tokenization/embedding of the text that we created earlier

to do this, we will use the langchain library, and Ollama gemma2 model, which can be downloaded thourgh our command prompt with this code 'ollama run gemma2:2b'¶

from langchain_community.document_loaders import TextLoader from langchain_text_splitters import RecursiveCharacterTextSplitter from langchain_community.embeddings import OllamaEmbeddings from langchain_chroma import Chroma

loader = TextLoader("text.txt") text_docs = loader.load()

splitter = RecursiveCharacterTextSplitter(chunk_size=100, chunk_overlap=30) final_docs = splitter.split_documents(text_docs)

myembeddings = OllamaEmbeddings(model="gemma2:2b")

vectordb = Chroma.from_documents( documents=final_docs, embedding=myembeddings, persist_directory="./chroma_db" )

as we can see in the above code in markdown, that first we will load the text, then split it then embed and finally store in chromadb, ready to be fed in our model¶

please remember, to run the code to downlaod gemma2 model, you need to have ollama.exe installed in your system locally¶

download it from https://ollama.com/download -¶

its a 745 mb file for me, so it should be less than a GB only depending on your OS, also the perk of downloading a small sized model is that you can run it locally without internet on your system¶

go to https://github.com/ollama/ollama/ to choose your preferred model than run the command given in your cmd¶

In [11]:

#final step-
#creating app.py file integrating the vectorized text embeddings and api key model from groq.com in the streamlit app interface

import os from dotenv import load_dotenv from langchain_community.embeddings import OllamaEmbeddings from langchain_chroma import Chroma from langchain_core.prompts import ChatPromptTemplate from langchain_groq import ChatGroq from langchain.chains import create_retrieval_chain from langchain.chains.combine_documents import create_stuff_documents_chain import streamlit as st

load_dotenv()

groq_api_key ="insert your key here" model = ChatGroq(model="gemma2-9b-it", groq_api_key=groq_api_key) embeddings = OllamaEmbeddings(model = 'gemma2:2b') mydb = Chroma(persist_directory='./chroma_db',embedding_function=embeddings) retriever = mydb.as_retriever(search_type='similarity', search_kwargs={"k":6})

st.title("Welcome to A's Caffe") query = st.chat_input("Ask me anything- ")

system_prompt= ( "You are an assitant for question answering task for a restaurant called A's caffe" "Use the following pieces of retrieved context to answer the question." "Make sure you talk very polite with the customer and don't write anything bad about the restaurant" "Your tone of reply should always be exciting and luring to the customers" "\n\n" "{context}" )

prompt = ChatPromptTemplate.from_messages([ ('system', system_prompt),('human',"{input}") ])

if query: question_answer_chain= create_stuff_documents_chain(model,prompt) rag_chain = create_retrieval_chain(retriever, question_answer_chain)

response = rag_chain.invoke({'input':query})
st.write(response['answer'])

In [ ]:

#in the code I have reproduced in the above cell, we can see that we have integrated the system and human prompt, using both the models

as you can see, I have removed the api_key used and we need to store it in a secret file named '.env', the 'dot' makes it private - access the api key visiting https://groq.com/

https://console.groq.com/playground

visit the api key page and create and copy it and save it somewhere as it will be not visible after it¶

In [13]:

#final words - after you have saved every file, go to the command prompt and type 'streamlit run app.py' it will open the app
# As fastai's founder, Jeremy Howard keeps saying and as is the nike's logo - 'just do it', will be explaining both code snippet in other post

the up and running app¶

In [ ]: