About the Competite reader
Competite helps small software companies keep an eye on a handful of competitors. When a user adds a company, we read a few of its public pages, the ones the user confirmed by hand, and never more than twelve.
What we fetch
- Only pages a user explicitly confirmed: typically the home, pricing, features and changelog pages.
- At most six pages per company per read, once a week, or once a month on our free plan.
- During setup, up to four pages: the homepage, robots.txt and sitemap.xml, to suggest pages for the user to confirm.
- On our top plan, one light daily read of each confirmed page, to notice a change the same day.
- Never more than thirty fetches an hour aimed at one site, across all our users combined.
How we identify ourselves
We run no crawler of our own. Every page, setup included, is fetched by our provider's reader and carries its user agent; the request never comes from a Competite server.
How to stop it
We read robots.txt during setup and do not suggest pages it disallows. To be excluded entirely, write to abuse@competite.com with the domain; we remove it from every account and refuse it going forward. We never bypass paywalls, logins or rate limits.