Firecrawl
A web data API for AI apps and agents: search the web, scrape a page, crawl a whole site or map its URLs, and get back clean Markdown, JSON or screenshots. It handles JavaScript-heavy pages, has SDKs in many languages and an MCP server, and its core is open source under AGPL-3.0 if you want to self-host.
189.3k (firecrawl/firecrawl)
GitHub · Oct 2026
AGPL-3.0 (SDKs MIT)
GitHub · Oct 2026
Tool
AI & Machine Learning
Firecrawl
An hour to try, then reference
Intermediate
Free to start; paid plans for volume
What Firecrawl is
Firecrawl describes itself as the web data API for AI agents. Raw HTML is a poor input for a language model: menus, adverts, scripts and layout markup waste tokens and confuse it. Firecrawl fetches pages, renders JavaScript when needed and returns clean content a model can use.
What it does
The main endpoints are:
- Scrape: one URL to Markdown, HTML, structured JSON or a screenshot.
- Crawl: follow links across a whole site and collect every page.
- Map: list the URLs on a site quickly, so you can choose what to crawl.
- Search: run a web search and get the full content of the results, not just links.
- Interact: click, fill forms and pull dynamic content from a page.
- Parse: convert local files such as PDF, DOCX and XLSX.
There are SDKs for Python, Node.js, Go, Java, Rust, Ruby, .NET, PHP and Elixir, and an MCP server so agents in Claude, code editors and other MCP clients can use it as a tool.
Hosted or self-hosted
You can use the hosted API or run it yourself. The main repository is licensed under AGPL-3.0, with the SDKs under MIT. AGPL has obligations if you modify it and offer it as a network service, so read the licence before building a commercial product on a self-hosted copy.
Use it responsibly
The project itself reminds users that respecting each website's policies is their responsibility. Check a site's terms and robots rules, rate-limit your crawls, prefer official APIs or data exports where they exist, and never collect personal data you have no right to process.
Prerequisites and time
An API key from the dashboard (light testing works without one, through its MCP server) and basic Python or JavaScript. Scraping a first page takes minutes.
Free or paid?
You can start for free; heavier use needs a paid plan. Self-hosting is free apart from your own servers.
Common questions
Firecrawl or Crawl4AI? Firecrawl is easiest as a hosted API with many SDKs; Crawl4AI is a free Python library you run yourself under Apache 2.0. Try both on the same pages.
Best for: RAG, research and agent projects that need clean text from websites without writing and maintaining your own crawler.
Ready to start?
Opens on Firecrawl in a new tab.
Stuck on something specific?
Writing only gets you so far. If you want an answer to your situation rather than the general case, book a session and we will work through it together. Sessions are free for approved Sefism members, and a few slots open each week.
Follow along
New writing, resources and project ideas land here first.