Job scraping using LinkedIn, Indeed, Bright Data, Google Sheets

LinkedIn & Indeed Job Scraper with Bright Data & Google Sheets Export

Overview

This n8n workflow automates the process of scraping job listings from both LinkedIn and Indeed platforms simultaneously, combining results, and exporting data to Google Sheets for comprehensive job market analysis. It integrates with Bright Data for professional web scraping, Google Sheets for data storage, and provides intelligent status monitoring with retry mechanisms.

Workflow Components

1. 📝 Trigger Input Form - Type: Form Trigger - Purpose: Initiates the workflow with user-defined job search criteria - Input Fields: - City (required) - Job Title (required) - Country (required) - Job Type (optional dropdown: Full-Time, Part-Time, Remote, WFH, Contract, Internship, Freelance) - Function: Captures user requirements to start the dual-platform job scraping process

2. 🧠 Format Input for APIs - Type: Code Node (JavaScript) - Purpose: Prepares and formats user input for both LinkedIn and Indeed APIs - Processing: - Standardizes location and job title formats - Creates API-specific input structures - Generates custom output field configurations - Function: Ensures compatibility with both Bright Data datasets

3. 🚀 Start Indeed Scraping - Type: HTTP Request (POST) - Purpose: Initiates Indeed job scraping via Bright Data - Endpoint: https://api.brightdata.com/datasets/v3/trigger - Parameters: - Dataset ID: gdlpfll7v5hcqtkxl6l - Include errors: true - Type: discovernew - Discover by: keyword - Limit per input: 2 - Custom Output Fields: - jobid, companyname, jobtitle, descriptiontext - location, salaryformatted, companyrating - applylink, url, dateposted, benefits

4. 🚀 Start LinkedIn Scraping - Type: HTTP Request (POST) - Purpose: Initiates LinkedIn job scraping via Bright Data (parallel execution) - Endpoint: https://api.brightdata.com/datasets/v3/trigger - Parameters: - Dataset ID: gdl4dx9j9sscpvs7no2 - Include errors: true - Type: discovernew - Discover by: keyword - Limit per input: 2 - Custom Output Fields: - jobpostingid, jobtitle, companyname, joblocation - jobsummary, jobemploymenttype, jobbasepayrange - applylink, url, jobposteddate, companylogo