URL to Markdown for LLMs: 7 Ways to Do It, Compared

Short answer: for a quick script, fetch the HTML and convert it with markdownify or html2text. If you only want the article, without menus and footers, use trafilatura. None of these run JavaScript, so for script-heavy pages, or when you'd rather not run anything yourself, use a hosted reader: Jina Reader (put r.jina.ai/ in front of the URL), Firecrawl, or browserfoundry's extraction and rendering endpoints. The right pick comes down to two questions: does the page need JavaScript, and do you want the whole page or just the main text?

Why markdown, and not just HTML?

Because raw HTML is mostly not content. Open the source of a typical news article: a few paragraphs of text, wrapped in navigation, cookie banners, inline scripts, styles, tracking tags, share buttons and a footer longer than the article. Send that to an LLM as-is and you pay for every <div> and every bit of minified JavaScript, while the model digs for the three paragraphs that matter.

Markdown keeps what carries meaning (headings, lists, tables, links, code) in a format models read fluently, and drops the rest. The same page often shrinks by an order of magnitude in tokens. That's cheaper, faster, and noticeably better answers, since the model's attention goes to content instead of chrome.

Rule of thumb. If a human would skip it while reading, your LLM should not be paying for it.

The DIY route: Python libraries

All four need you to fetch the HTML first. Plain requests is enough for server-rendered pages:

import requests

html = requests.get(
    "https://example.com/article",
    timeout=30,
    headers={"User-Agent": "Mozilla/5.0"},  # some sites reject the default python UA
).text

1. markdownify: converts everything, faithfully

Tag-by-tag conversion. Simple, predictable, and loyal to a fault: menus, footers and that "Subscribe to our newsletter!" box all come along.

from markdownify import markdownify

md = markdownify(html, heading_style="ATX")  # "# Heading" rather than underlines

Want it leaner? Strip tags before converting:

md = markdownify(html, heading_style="ATX", strip=["script", "style", "nav", "footer", "img"])

2. html2text: the veteran

The long-standing alternative, with switches to drop images or links and to stop hard-wrapping lines:

import html2text

h = html2text.HTML2Text()
h.ignore_images = True
h.body_width = 0  # no hard wraps; LLMs don't need 78-column text
md = h.handle(html)

3. trafilatura: finds the article for you

The one to reach for when you want the story, not the page. It detects the main content, throws away navigation, sidebars and boilerplate, and outputs markdown:

import trafilatura

downloaded = trafilatura.fetch_url("https://example.com/article")
md = trafilatura.extract(downloaded, output_format="markdown", include_links=True)

It shines on article-shaped pages: blogs, news, docs. On product pages, search results and dashboards, "main content" is a much harder guess, so check its output on your actual sites.

4. MarkItDown: one converter for everything

Microsoft's converter handles PDF, Word, PowerPoint, Excel and HTML through one interface, which is handy when your pipeline mixes web pages with documents:

from markitdown import MarkItDown

md = MarkItDown().convert("https://example.com/article").text_content

The library cheat sheet

The JavaScript problem

Every library above reads the HTML the server sends. On a single-page app, that HTML is often an empty <div id="root"> and a pile of script tags. There is nothing to convert. The tell is easy to spot: your markdown comes back as "Loading..." or "Please enable JavaScript."

The fix is to render the page in a headless browser first, then convert:

from playwright.sync_api import sync_playwright
from markdownify import markdownify

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto("https://example.com/app", wait_until="networkidle")
    md = markdownify(page.content(), heading_style="ATX")
    browser.close()

It works. It also means you now run browsers: memory, timeouts, crashed Chromium processes, bot checks, and at scale a fleet to babysit. That's the point where hosted services start earning their price (and if you need full interactive sessions, see what cloud browsers cost).

Gotcha. networkidle can hang forever on pages with analytics beacons or live chat widgets. If it times out, wait for a specific selector instead: page.wait_for_selector("article").

The hosted route

5. Jina Reader: the one-liner

The lowest-effort option on the internet: put https://r.jina.ai/ in front of any URL and markdown comes back.

curl https://r.jina.ai/https://example.com/article

Per Jina's own rate-limit table, Reader allows 20 requests a minute without a key and 500 with a free key, and every new key comes with 10 million free tokens; after that you pay per output token. Jina lists an average Reader latency of around 8 seconds, so it suits pipelines better than an agent with a user tapping their foot. Its search endpoint, s.jina.ai, needs a key.

6. Firecrawl: when you want the whole site

Firecrawl bills 1 credit per page for a basic scrape, with 1,000 free credits a month and no card. The Hobby plan is $19 a month ($16 billed yearly) for 5,000 credits. Its real party trick is crawling and mapping entire sites, not single pages.

curl -X POST https://api.firecrawl.dev/v2/scrape \
  -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/article", "formats": ["markdown"]}'

Structured formats (JSON extraction and friends) cost extra credits per page, so check Firecrawl's credit table before turning them on.

7. browserfoundry: extraction, rendering and search on one key

browserfoundry has two ways to read a page, both on paid plans:

• POST /api/v2/data takes a url (or a list of urls) and returns clean, parsed content, in about 2 seconds. 2 credits per page, half that on paid plans during the v2 beta.
• POST /api/v2/browsers/fetch renders the page in a fresh stealth browser, runs its JavaScript, and returns markdown by default (or HTML, text, or just the links). 2 credits flat, 2 MB cap, and a render that times out at 45 seconds is refunded. Add "proxy": "auto" for a residential IP on sites that block datacenters.

import os, requests

r = requests.post(
    "https://api.keirolabs.cloud/api/v2/browsers/fetch",
    headers={"Authorization": f"Bearer {os.environ['KEIRO_API_KEY']}"},
    json={"url": "https://example.com/app", "dump": "markdown"},
    timeout=60,
)
r.raise_for_status()
print(r.json())

At list prices, 1,000 pages costs 2,000 credits: roughly $1.60 to $4.80 depending on plan. The same key covers web search, and /api/v2/search/content searches and returns the top results' page text in one call (3 credits), which is often all a research agent needs.

Jina Reader vs Firecrawl vs browserfoundry

Five things that will bite you

1. Encoding. requests guesses the charset from headers, and sometimes guesses wrong. If you see é where é should be, set r.encoding = r.apparent_encoding before reading r.text.
2. Relative links. Converters keep href="/pricing" as-is, which means nothing to an LLM. Resolve them with urllib.parse.urljoin(base_url, href) if links matter.
3. Giant pages. A 50,000-word documentation page doesn't fit a small context window. Split on headings (## ) and send only the sections that match the question.
4. Tables. Some converters flatten complex tables into word soup. If tables carry the data you need, check them specifically.
5. Cookie walls. A page that shows a consent dialog before the content may convert to only the consent dialog. A rendering service, or a browser that clicks "accept," fixes it.

Which should you pick?

• A one-off script over pages you know are static: trafilatura for articles, markdownify for everything else.
• Mixed documents and pages: MarkItDown.
• JavaScript-heavy pages at low volume: Playwright plus a converter, or Jina Reader if you'd rather not run browsers.
• Crawling an entire site: Firecrawl.
• An agent that searches, then reads, then sometimes has to click: one API that does all three, so search, extraction and browsers share a key and a bill.

How we put this together

The library usage follows each project's documentation: markdownify, html2text, trafilatura and MarkItDown. Limits and prices come from the providers' own pages as published in October 2026: Jina Reader and its rate-limit table, Firecrawl pricing, and browserfoundry's pricing and API docs. Per-1,000-page costs are arithmetic on those list prices. Check the live pages before you commit, since prices change.