Scrape ProductHunt using Google Gemini

Workflow Description: Product Data Extractor

This workflow automates the extraction of product data from Product Hunt by combining webhook interactions, HTML processing, AI-based data analysis, and structured output formatting. It is designed to handle incoming requests dynamically and return detailed JSON responses for further usage.

Overview

The workflow processes a product name submitted through a webhook. It fetches the corresponding Product Hunt page, extracts and analyzes inline scripts, and structures the data into a well-defined JSON format using AI tools. The final JSON response is returned to the client through the webhook.

Workflow Steps

1. Webhook Listener

- Node: Receive Product Request - Function: Captures incoming requests containing the product name to process. - Details: Accepts HTTP requests and extracts the product parameter from the query string, such as <customwebhookurl>/?product=epigram.

2. Fetch Product HTML