Crawl documentation sites and build an AI knowledge base with Olostep

AI Documentation Crawler & Knowledge Base Builder

This n8n template automatically crawls technical documentation websites, scrapes their content, and converts it into clean, structured, developer-friendly documentation. Each page is organized into folders and saved as Google Docs, making it easy to build or maintain an internal knowledge base.

Who’s it for - Developer teams maintaining internal or external documentation - SaaS companies onboarding users or support teams - AI builders creating documentation-based knowledge bases - Anyone who wants to turn raw docs into structured, readable references

How it works / What it does 1. Manual Trigger - The workflow starts manually whenever you want to crawl or refresh documentation.

2. Documentation Discovery (Crawler) - The workflow crawls a root documentation URL and generates a sitemap of all discoverable documentation pages.

3. URL Processing - The sitemap is split into individual URLs. - The workflow dynamically analyzes URL depth to recreate the documentation hierarchy.

4. Folder Structure Creation - A parent folder is created in Google Drive for the service. - Subfolders are automatically generated to mirror the documentation structure (based on URL paths).

5. Content Scraping - Each documentation page is scraped using the Olostep API. - Clean markdown content is extracted from the page.