Operated by Sourcelane✓ verified
Scrape all posts from any Substack publication. Title, publish date, likes, comments, restacks, word count, paywall status, and author for every post in the archive.
A web-scraping connector that extracts complete post-level metadata from Substack publications (supports Substack subdomains and custom domains). It collects publication information (name, URL, description), per-post identifiers and URLs, content metadata (titles, subtitles, descriptions, content type, cover image), publishing data (publish timestamps and word counts), paywall status, engagement metrics (likes, comments, restacks), author details (name and handle), and a scrape timestamp for each post; outputs one record per post optimized for downstream analysis and ranking by engagement or cadence.
Verified Sep 28, 3:55 AM
$0.0018
per post
Up to 100 per call. Only pay for results returned; failed calls are refunded.
Content research
Analyze topics and formats published by a newsletter
Competitor analysis
Benchmark newsletters within a niche
Publishing cadence tracking
Measure posting frequency and timing
Newsletter sponsorship research
Assess potential sponsorship targets by audience engagement
Top-performing post discovery
Identify posts with high likes, comments, or restacks
Creator and influencer research
Build lists of active Substack writers in any vertical
Substack publication URLs to scrape.
Optional settings
| Window | Uptime | Success rate | p50 | p95 | Calls |
|---|---|---|---|---|---|
| 24h | — | — | — | — | 0 |
| 7d | — | — | — | — | 0 |
| 30d | — | — | — | — | 0 |
Call it through Sourcelane's gateway or MCP server. We run the connector, apply your agent's spend guardrails, and bill only the results returned.
Call it via Sourcelane
curl -X POST https://api.usesourcelane.com/v1/call \
-H "Authorization: Bearer sl_live_your_agent_key" \
-H "Content-Type: application/json" \
-d '{"listing":"substack-post-scraper","params":{"urls":["https://astralcodexten.substack.com","https://www.thefitzwilliam.com"],"maxItems":5},"maxResults":10}'Request params
| Field | Type | Required | Description |
|---|---|---|---|
urls | array | required | Substack publication URLs to scrape. Accepts substack.com subdomains (https://platformer.substack.com) and custom domains (https://platformer.news). All posts are returned newest-first. |
maxItems | integer | optional | Maximum number of posts to collect across all publications. Set to 0 for unlimited. Capped at 100 per call. |
Each result contains
| Field | Type | Description |
|---|---|---|
publicationName | null | Publication Name |
publicationUrl | string | Publication Url |
publicationDescription | null | Publication Description |
postId | integer | Post Id |
postUrl | string | Post Url |
title | string | Title |
subtitle | string | Subtitle |
description | string | Description |
publishedAt | string | Published At |
type | string | Type |
isPaywalled | boolean | Is Paywalled |
likesCount | integer | Likes Count |
commentsCount | integer | — |
restacksCount | integer | — |
wordCount | integer | — |
coverImageUrl | string | — |
authorName | string | — |
authorHandle | null | — |
scrapedAt | string | — |
Pick your tool and connect in under a minute. Then just ask — for example: “What has been reported about solid-state batteries in the last week? Cite sources.”
Connect Claude
Web, desktop and mobile. One click, then confirm.
Adds the Sourcelane MCP server with your key as a header.
claude mcp add --transport http sourcelane https://api.usesourcelane.com/mcp \
--header "Authorization: Bearer sl_live_your_agent_key"Alternative to the connector URL: add to claude_desktop_config.json, then restart Claude.
{
"mcpServers": {
"sourcelane": {
"command": "npx",
"args": [
"-y",
"mcp-remote",
"https://api.usesourcelane.com/mcp",
"--header",
"Authorization:${AUTH_HEADER}"
],
"env": {
"AUTH_HEADER": "Bearer sl_live_your_agent_key"
}
}
}
}Add to ~/.cursor/mcp.json (or Windsurf's mcp_config.json).
{
"mcpServers": {
"sourcelane": {
"url": "https://api.usesourcelane.com/mcp",
"headers": {
"Authorization": "Bearer sl_live_your_agent_key"
}
}
}
}Save as .vscode/mcp.json. VS Code prompts for your key once.
{
"servers": {
"sourcelane": {
"type": "http",
"url": "https://api.usesourcelane.com/mcp",
"headers": {
"Authorization": "Bearer ${input:sourcelane-key}"
}
}
},
"inputs": [
{
"type": "promptString",
"id": "sourcelane-key",
"description": "Sourcelane agent key",
"password": true
}
]
}Create a GPT → Actions → Import from URL, then Authentication: API Key, Bearer.
https://usesourcelane.com/openapi.jsonOne POST. Pass maxResults to cap cost.
curl -X POST https://api.usesourcelane.com/v1/call \
-H "Authorization: Bearer sl_live_your_agent_key" \
-H "Content-Type: application/json" \
-d '{"listing":"substack-post-scraper","params":{"urls":["https://astralcodexten.substack.com","https://www.thefitzwilliam.com"],"maxItems":5},"maxResults":10}'Wrap the REST call as a tool, or point an MCP client at the endpoint.
import requests
res = requests.post(
"https://api.usesourcelane.com/v1/call",
headers={"Authorization": "Bearer sl_live_your_agent_key"},
json={
"listing": "substack-post-scraper",
"params": {"urls":["https://astralcodexten.substack.com","https://www.thefitzwilliam.com"],"maxItems":5},
"maxResults": 10,
},
timeout=300,
)
body = res.json()
print(body["receipt"]["chargedMicros"], "micro-USD for", body["receipt"]["results"], "results")
print(body["data"])—
0 reviews
Leave a review
Loading reviews…