Ai website scraper & company intelligence

AI Website Scraper & Company Intelligence

Description

This workflow automates the process of transforming any website URL into a structured, intelligent company profile. It's triggered by a form, allowing a user to submit a website and choose between a "basic" or "deep" scrape.

The workflow extracts key information (mission, services, contacts, SEO keywords), stores it in a structured Supabase database, and archives a full JSON backup to Google Drive. It also features a secondary AI agent that automatically finds and saves competitors for each company, building a rich, interconnected database of company intelligence.

---

Quick Implementation Steps

1. Import the Workflow: Import the provided JSON file into your n8n instance. 2. Install Custom Community Node: You must install the community node from: [https://www.npmjs.com/package/n8n-nodes-crawl-and-scrape](https://www.npmjs.com/package/n8n-nodes-crawl-and-scrape) FIRECRAWL N8N Documentation [https://docs.firecrawl.dev/developer-guides/workflow-automation/n8n](https://docs.firecrawl.dev/developer-guides/workflow-automation/n8n)

3. Install Additional Nodes: n8n-nodes-crawl-and-scrape and n8n-nodes-mcp fire crawl mcp . 4. Set up Credentials: Create credentials in n8n for FIRE CRAWL API,Supabase, Mistral AI, and Google Drive. 5. Configure API Key (CRITICAL): - Open the Web Search tool node. - Go to Parameters → Headers and replace the hardcoded Tavily AI API key with your own. 6. Configure Supabase Nodes: - Assign your Supabase credential to all Supabase nodes. - Ensure table names (e.g., companies, competitors) match your schema. 7. Configure Google Drive Nodes: - Assign your Google Drive credential to the Google Drive2 and save to Google Drive1 nodes. - Select the correct Folder ID. 8. Activate Workflow: Turn on the workflow and open the Webhook URL in the “On form submission” node to access the form.