DoNoHarm Crawler

You might have arrived here since you found an entry in your web server logs with a user agent string like this:

DoNoHarm Crawler/1.0 (+https://crawler.donoharm.world)

or noticed your website was visited from an IP with a reverse DNS entry of crawler.donoharm.world. If so, then your site was indexed by the web crawler of our search engine. The crawler is run by DoNoHarm GmbH.

If this is unexpected, please let us know and we'll look into the matter further. If you did not add your site yourself and would like it to be removed, we'll be happy to do it for you. You can contact us at support@donoharm.world.

Public IP addresses

For firewall allowlists and bot verification, use our public IP list (JSON). It contains our current public IPv4 and IPv6 addresses and a revision identifier. No authentication is required.

Products using this crawler

The crawler is used by Monocle Search and SearchCue.

We do our best to behave well:

We attempt to crawl no more than 5 pages per second (unless we believe your site to be hosted by Squarespace, another hosting provider we have an agreement with, or you have explicitly given us permission to do so)
We do our best to not re-download pages that have not changed