Setup & Installation
What This Skill Does
Fetches the raw HTML of any URL through the Crawlbase MCP server's crawl tool, with JavaScript rendering, smart proxy rotation, and anti-bot handling done for you. Supports device emulation, geo-targeted requests, AJAX and page-load wait tuning, optional screenshot capture, and pushing results straight to Crawlbase Cloud Storage.
Your agent gets the rendered HTML of hard targets without you operating proxies or headless browsers — requests ride Crawlbase infrastructure trusted by 70,000+ developers.
When to use it
- Inspecting the full HTML structure of a page before writing extraction logic
- Fetching JavaScript-rendered pages that return empty bodies to plain HTTP clients
- Comparing how a page renders for mobile versus desktop browsers
- Geo-targeting a crawl to see country-specific pricing or content
- Storing crawl results in Crawlbase Cloud Storage for later retrieval
More from Crawlbase
The MCP server behind these skills (npm @crawlbase/mcp). Connects AI agents and LLMs to real-time web data with JavaScript rendering, automatic proxy rotation, and anti-bot protection. Works with Claude, Cursor, and Windsurf.
LangChain document loader, tool, and retriever backed by the Crawlbase Crawling API. Returns clean Markdown documents ready for chunking (pip install langchain-crawlbase).
Native Crawlbase node for n8n. Adds Crawling API credentials and options like JS rendering, geo-targeting, screenshots, and storage to your workflows without hand-wiring HTTP Request nodes.