File to Markdown Crawl Powered by Cloudflare

Markdown for Agents

The web was built for humans. AI agents need structured data. We convert any URL to clean Markdown using Cloudflare's native text/markdown content type — no parsing, no waste.

80% fewer tokens than raw HTML
⚡ 0.1–0.6s response time
🔒 No signup required
https://markdown.new/any-url-here

Prepend markdown.new/ to any URL in your browser for instant conversion

Three-Tier Conversion Pipeline

We try the fastest method first and fall back automatically. Every request gets the best possible Markdown.

Primary

Markdown for Agents

Fetches the URL with Accept: text/markdown. Cloudflare-enabled sites return clean Markdown directly from the edge — zero parsing needed.

fetch(url, { headers: { Accept: "text/markdown" } })
Fallback 1

Workers AI

If content negotiation returns HTML, we pass it directly through Cloudflare Workers AI toMarkdown() — fast, no re-fetch needed.

env.AI.toMarkdown([{ name: "page.html", blob: htmlBlob }])
Fallback 2

Browser Rendering

For JS-heavy pages, we render in a headless browser via Cloudflare's Browser Rendering API for full page content extraction.

POST /browser-rendering /markdown { "url": "https://..." } // Full page render

Why Markdown Matters for AI

Feeding raw HTML to an AI is like paying by the word to read packaging instead of the letter inside.

HTML — 12-15 tokens
<h2 class="section-title" id="about">About Us</h2>
16,180 tokens for a blog post
Markdown — 3 tokens
## About Us
3,150 tokens for the same post
80%
Token reduction
5x
More content per context window
0
External dependencies

Conversion Options

Control the conversion method and image handling via query parameters or POST body.

Parameter Values Default
method auto ai browser auto
retain_images true false false
Browser Rendering (JS-heavy sites)
curl -s 'https://markdown.new/' \ -H 'Content-Type: application/json' \ -d '{"url": "https://example.com", "method": "browser"}'
URL with query parameters
https://markdown.new/https://example.com?method=browser&retain_images=true
Workers AI with images retained
curl -s 'https://markdown.new/' \ -H 'Content-Type: application/json' \ -d '{"url": "https://example.com", "method": "ai", "retain_images": true}'

What You Get Back

Clean Markdown with metadata headers, including token count via x-markdown-tokens.

HTTP/2 200
content-type: text/markdown; charset=utf-8 x-markdown-tokens: 725 // estimated token count vary: accept --- title: Markdown for Agents --- # Introducing Markdown for Agents The way content and businesses are discovered online is changing rapidly. Now the traffic is increasingly coming from AI crawlers and agents that demand structured data...

Built For

Developers, AI agents, and teams building the next generation of intelligent applications.

→ AI agents that browse and summarize the web
→ RAG pipelines that need clean document chunks
→ Training data preparation for LLMs
→ Documentation migration and static site generators
→ Knowledge base builders and research tools
→ Content archival in human-readable format

Crawl Entire Websites

Need more than one page? Crawl to Markdown converts entire site sections into a single Markdown file.

🕸 Crawl up to 100 pages per job with automatic link discovery
⚡ Async job-based — start a crawl, track progress, download when done
📦 Download all pages as a single .md file or copy to clipboard
⚙ Configure depth, page limit, and JS rendering per crawl
Try Crawl to Markdown

Frequently Asked Questions

Everything you need to know about markdown.new.

What is markdown.new?
It converts any public URL into clean, structured Markdown optimized for LLMs and AI agents. It strips away HTML bloat (ads, scripts, styling) while preserving headings, lists, links, and tables — making web content AI-ready with up to 80% fewer tokens.
How do I use it?
Three ways, no signup needed. Browser: prepend https://markdown.new/ to any URL. cURL/API: POST a JSON body with {"url": "https://example.com"}. Options: add method (auto, ai, browser) and retain_images (true/false) as query params or in the POST body.
Is it free?
Yes, always. No subscriptions, no paywalls. It’s an open utility for developers, AI builders, and teams. Just respect the fair usage limits so the service stays reliable for everyone.
What are the rate limits?
500 requests per day per IP address. If you exceed the limit, you’ll receive an HTTP 429 response. Check the x-rate-limit-remaining header to track your usage. For heavier workloads, consider self-hosting the open-source logic.
How does the conversion pipeline work?
It uses a three-tier fallback chain. Tier 1: requests the URL with Accept: text/markdown for native Markdown. Tier 2: passes HTML through Cloudflare Workers AI toMarkdown(). Tier 3: renders JS-heavy pages in a headless browser via Cloudflare Browser Rendering. The fastest successful tier wins.
Are there any limitations?
Public URLs only — paywalled or authenticated pages won’t work. JS-heavy sites using the browser method add ~1–2s of latency. Images are excluded by default (enable with retain_images=true). Very large pages may be truncated.
Is it legal to convert web pages?
Yes, for public pages. markdown.new uses standard HTTP requests (like your browser) and doesn’t store or republish content. Always respect each site’s Terms of Service and robots.txt. Avoid mass-scraping copyrighted content for redistribution.
Can I crawl an entire website?
Yes! Use /crawl to crawl up to 100 pages from any site. It uses Cloudflare’s Browser Rendering /crawl API with async job tracking. You can configure page limits (up to 100), crawl depth (up to 10), and enable JS rendering. Results are stored for 14 days.
How can I block this service from my site?
Our bot identifies as markdown.new/1.0. Add this to your robots.txt: User-agent: markdown.new Disallow: /. You can also block by User-Agent at the WAF/server level, use a llms.txt file to signal AI usage preferences, or return X-Robots-Tag: noindex headers.