Setup & Installation
What This Skill Does
Fetches up to 100 stored pages from Crawlbase Cloud Storage in a single round-trip, given a list of request ids (typically from storage-list). Bodies come back as metadata only, full JSON, raw HTML, or converted Markdown, and auto_delete enables a fetch-and-drain pattern. All rids in one call must live in the same token silo.
One call instead of a hundred: batch retrieval keeps token usage and latency down when an agent processes stored crawls at scale.
When to use it
- Pulling the bodies of dozens of stored crawls in one round-trip
- Draining a storage queue by fetching items and deleting them afterwards
- Getting lightweight metadata for stored items before requesting full bodies
- Bulk-converting stored pages to Markdown for a RAG ingestion job
More from Crawlbase
The MCP server behind these skills (npm @crawlbase/mcp). Connects AI agents and LLMs to real-time web data with JavaScript rendering, automatic proxy rotation, and anti-bot protection. Works with Claude, Cursor, and Windsurf.
LangChain document loader, tool, and retriever backed by the Crawlbase Crawling API. Returns clean Markdown documents ready for chunking (pip install langchain-crawlbase).
Native Crawlbase node for n8n. Adds Crawling API credentials and options like JS rendering, geo-targeting, screenshots, and storage to your workflows without hand-wiring HTTP Request nodes.