DoNoHarm Crawler
You might have arrived here since you found an entry in your web server logs with a user agent string like this:
DoNoHarm Crawler/1.0 (+https://crawler.donoharm.world)
or noticed your website was visited from an IP with a reverse DNS entry of crawler.donoharm.world.
If so, then your site was indexed by the web crawler of our search engine. If this is unexpected,
then please let us know and we'll look into the matter further. We only crawl sites that we were
explicitly asked to index by our customers. If you did not add your site yourself
and would like it to be removed, we'll be happy to do it for you. You can contact us at support@donoharm.world.
Products using this crawler
The crawler is run by DoNoHarm - Bauer & Eide GbR. It is used by Monocle Search and Winnow Search.
We do our best to behave well:
- We attempt to crawl no more than 5 sites per second (unless we believe your site to be hosted by Squarespace in which case we crawl significantly faster, or you have explicitly given us permission to do so)
- We do our best to not re-download pages that have not changed