Build a company website RAG chatbot using Apify, Pinecone and Gemini

AI chatbots are only as good as the data they learn from. Most large language models (LLM) rely only on their training datasets.

If you want the chatbots to know more about your business, the best is to implement a retrieval-augmented generation (RAG) pipeline to train Gemini with your website data. This is what this workflow will help you to do.

This workflow uses a scheduler to scrape a website on a regular basis using [Apify](https://apify.com/); web pages are then indexed or updated in a [Pinecone](https://www.pinecone.io/) vector database. This allows the chatbot to provide accurate and up-to-date information. The workflow uses [Google's Gemini AI](https://aistudio.google.com/) for both embeddings and response generation. How does it work? This workflow is split into 2 sub-logics highlighted with green sticky notes: RAG Training logic Chatbot logic RAG training logic 1. Use the [Apify Website Content Crawler](https://apify.com/apify/website-content-crawler) to retrieve all content from your website 2. The Pinecone Vector Store node indexes the text chunk in a Pinecone index. 3. The Embeddings Google Gemini node generates embeddings for each text chunk Chatbot logic 1. The Chat Trigger node receives user questions through a chat interface. An AI Agent node handles those requests. 1. The AI Agent node uses a Vector Store Tool node, linked to a Pinecone Vector Store node in query mode, to retrieve relevant text chunks from Pinecone based on the user's question. 2. The AI Agent sends the retrieved information and the user's question to the Google Gemini Chat Model (gemini-pro). How to set up this template? All nodes with an orange sticky note require setup.

Get your tools set up: 1 Google Cloud Project and Vertex AI API: Create a Google Cloud project. Enable the Vertex AI API for your project. Obtain a Google AI API key from Google AI Studio

2 Get an Apify account Create an [Apify account](https://apify.com/)

3 Pinecone Account: Create a free account on the Pinecone website. Obtain your API key from your Pinecone dashboard. Create an index named company-website in your Pinecone project.

Configure credentials in your n8n environment for: Google Gemini(PaLM) Api (using your Google AI API key) Pinecone API (using your Pinecone API key)

Setup trigger frequency: Edit the Schedule Trigger to match the frequency at which you wish to update your RAG If you want to train your chatbot only once, you can replace it with a click trigger.