Amazon Product Search Scraper

Overview

The Amazon Product Search Scraper is a robust, automated workflow designed to streamline the extraction of structured product data from Amazon search results into Google Sheets. It intelligently retrieves raw HTML from Amazon search result pages, sanitises it to preserve only the necessary product elements, and employs a large language model to accurately extract key details such as product names, descriptions, ratings, review counts, and pricing information.

This fully automated pipeline integrates several powerful tools: Google Sheets is used to manage input URLs and capture output; BrightData fetches the HTML content of the search pages; a custom n8n Function node cleans the HTML to isolate product-specific elements; and LangChain, powered by OpenRouter GPT-4, parses and formats the data. The final structured dataset is then automatically saved back into Google Sheets for seamless access and analysis.

Beyond Amazon, this flexible architecture can be adapted to scrape data from other major e-commerce platforms including eBay, Walmart, and Etsy making it a versatile solution for a wide range of product intelligence needs.

Features & Benefits

• End-to-End Automation: From input to output, every step from scraping to structuring and storing product data is completely automated.
• Structured Data Extraction: Extracts product name, description, price, rating, and number of reviews in a structured format.
• Google Sheets Integration: Automatically pulls input URLs and pushes final data to Google Sheets for ease of use and collaboration.
• Scalable Scraping: Handles multiple search result pages simultaneously, ideal for large-scale product monitoring.
• Cross-Platform Compatibility: Built to be extended to other e-commerce sites such as eBay, Walmart, and Etsy.
• Custom HTML Cleaning: A specialised n8n function node ensures only the most relevant HTML elements are retained, boosting LLM parsing accuracy.
• LLM-Driven Parsing: Uses GPT-4 via LangChain and OpenRouter to semantically understand and extract product information accurately.

Use Cases

• E-commerce Analysts: Continuously track competitor pricing, ratings, and inventory changes across Amazon and other marketplaces.
• Market Researchers: Collect product data for trend analysis, customer sentiment, and category-level insights.
• Data Teams: Integrate clean, structured product data directly into BI tools or data lakes.
• Affiliate Marketers: Automatically update affiliate catalogues with current product details, pricing, and availability.
• Retail Intelligence Units: Maintain a pulse on market activity across multiple platforms.

Challenge & Results

Prior to implementation, businesses often relied on manual copy-paste methods or unreliable scraping tools to collect Amazon product data leading to inconsistent results and wasted time. This posed significant challenges in scaling market intelligence efforts and maintaining data quality.

By deploying the Amazon Product Search Scraper, our client was able to automate the entire process end-to-end, reducing manual labour by over 90%. The structured outputs improved data reliability and fed directly into existing analytics dashboards. Furthermore, the adaptable architecture enabled the client to quickly repurpose the workflow for use on eBay and Walmart, extending its value across multiple sales channels.