Setup & Installation
What This Skill Does
Extracts clean, LLM-ready Markdown from any URL through the crawl_markdown tool of the Crawlbase MCP server. Ads, navigation, and boilerplate are stripped so only the main content comes back — ready for context windows, summaries, and RAG pipelines. Supports device emulation, custom user agents, and storing results in Crawlbase Cloud Storage.
You skip the HTML-stripping step entirely: the agent asks for a URL and gets back Markdown it can reason over directly, even from JavaScript-heavy pages.
When to use it
- Summarizing articles, blog posts, and news pages without HTML noise
- Feeding documentation and changelogs into a model's context window
- Ingesting web content into a vector store for RAG pipelines
- Monitoring competitor product pages as readable Markdown
- Storing extracted Markdown in Crawlbase Cloud Storage for later use
More from Crawlbase
The MCP server behind these skills (npm @crawlbase/mcp). Connects AI agents and LLMs to real-time web data with JavaScript rendering, automatic proxy rotation, and anti-bot protection. Works with Claude, Cursor, and Windsurf.
LangChain document loader, tool, and retriever backed by the Crawlbase Crawling API. Returns clean Markdown documents ready for chunking (pip install langchain-crawlbase).
Native Crawlbase node for n8n. Adds Crawling API credentials and options like JS rendering, geo-targeting, screenshots, and storage to your workflows without hand-wiring HTTP Request nodes.