Upsert huge documents in a vector store with Supabase and Notion

Purpose

This workflow adds the capability to build a RAG on living data. In this case Notion is used as a Knowledge Base. Whenever a page is updated, the embeddings get upserted in a Supabase Vector Store.

It can also be fairly easily adapted to PGVector, Pinecone, or Qdrant by using a custom HTTP request for the latter two.

Demo

[](https://youtu.be/ELAxebGmspY)

How it works

- A trigger checks every minute for changes in the Notion Database. The manual polling approach improves accuracy and prevents changes from being lost between cached polling intervals. - Afterwards every updated page is processed sequentially - The Vector Database is searched using the Notion Page ID stored in the metadata of each embedding. If old entries exist, they are deleted. - All blocks of the Notion Database Page are retrieved and combined into a single string - The content is embedded and split into chunks if necessary. Metadata, including the Notion Page ID, is added during storage for future reference. - A simple Question and Answer Chain enables users to ask questions about the embedded content through the integrated chat function

Prerequisites