Back to MCP servers

polite-crawler

MCP serverMCP registryUnknown

A polite crawler for agents: obeys robots.txt, fetches each page once, finds and verifies quotes.

Description provenance: Original source

Open canonical source

7 observations

Domains & Uses

Domains
Agents, skills and MCP

Installation

claude mcp add polite-crawler -- npx polite-crawler-mcp

Check at the source

Compatibility and prerequisites must be checked in the official documentation.

https://github.com/mark1russell7/crawler

Classifications

Nature
Tool

Measurement history

Comparable readings are grouped by signal and unit.

No comparable history is available.