Shopify Store Product Scraper

E-commerce & Marketplaces

Operated by Sourcelane✓ verified

A Shopify-focused web scraper that extracts full product catalogs from Shopify storefronts into structured records: product metadata (store domain, product identifiers, handles, titles, vendor, product type, tags, product URLs, creation/updated/published dates), clean plain-text product descriptions (HTML stripped), pricing summaries (minimum/maximum across variants and a sale flag when any variant is discounted), per-variant details (variant identifiers, variant titles, SKU, price and compare-at price, availability, option values), and media (featured image, image count, all image URLs). Technical capabilities include reading the Shopify public products.json endpoint, configurable output granularity (one record per product or one per variant), collection-level scoping, automated pagination across a store catalog with a resumable next-page cursor per store, and structured export-ready records for downstream e-commerce analysis.

Verified Sep 28, 12:10 AM

$0.005

per result

Up to 100 per call. Only pay for results returned; failed calls are refunded.

What people use it for

Competitor price monitoring — snapshot a rival store's catalog and prices on a schedule
Dropshipping & product research — discover what's selling, at what price, and in which variants
Catalog migration — pull an entire catalog before moving platforms
Market & assortment analysis — compare vendors, product types, and price bands across stores
Deal & discount tracking — flag products on sale via a sale indicator

Trust & reliability

7d uptime trend

Window	Uptime	Success rate	p50	p95
24h	—	—	—	—
7d	—	—	—	—
30d	—	—	—	—

Calling contract

Call it through Sourcelane's gateway or MCP server. We run the connector, apply your agent's spend guardrails, and bill only the results returned.

Call it via Sourcelane

curl -X POST https://cool-puffin-608.convex.site/v1/call \
  -H "Authorization: Bearer sl_live_your_agent_key" \
  -H "Content-Type: application/json" \
  -d '{"listing":"shopify-store-product-scraper","params":{"storeUrls":["allbirds.com"],"maxItems":5},"maxResults":10}'

Request params

Field	Type	Required	Description
`storeUrls`	array	required	One or more Shopify store domains (e.g. allbirds.com) or full URLs (e.g. https://www.gymshark.com). Non-Shopify or unsupported stores are skipped with a warning.
`collection`	string	optional	Restrict the scrape to a single collection handle (e.g. 'mens-shoes' from /collections/mens-shoes). Leave empty to fetch the entire catalog. Applies to every store in the list.
`includeVariants`	boolean	optional	When on, emit one results row per product variant (size/color/etc.). When off, emit one row per product with aggregated price range and variant count.
`maxItems`	integer	optional	Maximum number of rows to push across all stores combined. Set 0 for unlimited (paginates until every store is exhausted or the run nears its timeout). Capped at 100 per call.
`pageId`	string	optional	Paste the NEXT_PAGE_ID value from the previous run's Key-value store to resume where you left off. A JSON map of store domain to the next page number.

Each result contains

Field	Type	Description
`storeDomain`	string	Store Domain
`productId`	integer	Product Id
`handle`	string	Handle
`title`	string	Title
`vendor`	string	Vendor
`productType`	string	Product Type
`tags`	array	Tags
`productUrl`	string	Product Url
`featuredImage`	string	Featured Image
`imageCount`	integer	Image Count
`imageUrls`	array	Image Urls
`bodyText`	string	Body Text
`publishedAt`	string	—
`createdAt`	string	—
`updatedAt`	string	—
`priceMin`	integer	—
`priceMax`	integer	—
`isOnSale`	boolean	—
`variantCount`	integer	—
`variantId`	integer	—
`variantTitle`	string	—
`sku`	string	—
`price`	integer	—
`compareAtPrice`	null	—
`available`	boolean	—
`position`	integer	—
`options`	array	—
`variantImage`	null	—

Use Shopify Store Product Scraper from Claude, ChatGPT or Cursor

Pick your tool and connect in under a minute. Then just ask — for example: “Track the price of these 20 grocery items at Woolworths and Coles every week.”

Connect Claude

Web, desktop and mobile. Paste one URL.

1Copy your personal connector URL
2In Claude open Settings → Connectors → Add custom connector
3Paste the URL and click Add — done

Manual setup (config files, REST, Python) +

Claude Code

Adds the Sourcelane MCP server with your key as a header.

terminal

claude mcp add --transport http sourcelane https://cool-puffin-608.convex.site/mcp \
  --header "Authorization: Bearer sl_live_your_agent_key"

Claude Desktop (config file)

Alternative to the connector URL: add to claude_desktop_config.json, then restart Claude.

claude_desktop_config.json

{
  "mcpServers": {
    "sourcelane": {
      "command": "npx",
      "args": [
        "-y",
        "mcp-remote",
        "https://cool-puffin-608.convex.site/mcp",
        "--header",
        "Authorization:${AUTH_HEADER}"
      ],
      "env": {
        "AUTH_HEADER": "Bearer sl_live_your_agent_key"
      }
    }
  }
}

Cursor & Windsurf

Add to ~/.cursor/mcp.json (or Windsurf's mcp_config.json).

mcp.json

{
  "mcpServers": {
    "sourcelane": {
      "url": "https://cool-puffin-608.convex.site/mcp",
      "headers": {
        "Authorization": "Bearer sl_live_your_agent_key"
      }
    }
  }
}

VS Code

Save as .vscode/mcp.json. VS Code prompts for your key once.

.vscode/mcp.json

{
  "servers": {
    "sourcelane": {
      "type": "http",
      "url": "https://cool-puffin-608.convex.site/mcp",
      "headers": {
        "Authorization": "Bearer ${input:sourcelane-key}"
      }
    }
  },
  "inputs": [
    {
      "type": "promptString",
      "id": "sourcelane-key",
      "description": "Sourcelane agent key",
      "password": true
    }
  ]
}

ChatGPT Custom GPT (Actions)

Create a GPT → Actions → Import from URL, then Authentication: API Key, Bearer.

OpenAPI schema URL

https://sourcelane-amber.vercel.app/openapi.json

REST

One POST. Pass maxResults to cap cost.

curl

curl -X POST https://cool-puffin-608.convex.site/v1/call \
  -H "Authorization: Bearer sl_live_your_agent_key" \
  -H "Content-Type: application/json" \
  -d '{"listing":"shopify-store-product-scraper","params":{"storeUrls":["allbirds.com"],"maxItems":5},"maxResults":10}'

Python, LangChain, CrewAI, OpenAI Agents SDK…

Wrap the REST call as a tool, or point an MCP client at the endpoint.

python

import requests

res = requests.post(
    "https://cool-puffin-608.convex.site/v1/call",
    headers={"Authorization": "Bearer sl_live_your_agent_key"},
    json={
        "listing": "shopify-store-product-scraper",
        "params": {"storeUrls":["allbirds.com"],"maxItems":5},
        "maxResults": 10,
    },
    timeout=300,
)
body = res.json()
print(body["receipt"]["chargedMicros"], "micro-USD for", body["receipt"]["results"], "results")
print(body["data"])

Reviews

—

0 reviews

Leave a review

Loading reviews…