# TheCrawler

Universal web scraper with LLM-ready markdown, RAG chunking, PDF/DOCX support.

- Weekly downloads: 604
- GitHub stars: 2
- Health: at-risk
- Repository: https://github.com/manchittlab/thecrawler
- Registry names: io.github.manchittlab/thecrawler
- Categories: Browser & web scraping, Files & documents, Search & knowledge
- Snapshot: 2026-10-27

## Install

```sh
npx -y thecrawler
```

```json
{
  "mcpServers": {
    "thecrawler": {
      "command": "npx",
      "args": [
        "-y",
        "thecrawler"
      ]
    }
  }
}
```

Source: https://mcpmorning.com/mcp/manchittlab-thecrawler/