HeiLocalBot

HeiLocalBot is the web crawler of Hei!Local — the semantic search engine for Romanian online shops. It discovers and refreshes products from Romanian e-commerce so they can be found in search.
Citește această pagină în română →

Identification (User-Agent)

HeiLocalBot identifies itself on every HTTP request through the following User-Agent header:

You can recognise our traffic by the HeiLocalBot token in the User-Agent.

Registered IP address

All HeiLocalBot requests originate from the following IP address. You can use it to confirm that the traffic is genuinely ours (and not a bot impersonating our User-Agent):

Cryptographic verification (Web Bot Auth)

In addition to the IP address, HeiLocalBot cryptographically signs every request following the Web Bot Auth standard (HTTP Message Signatures, RFC 9421, Ed25519 key). This lets you confirm a request is genuinely ours even if our IP address changes — a bot merely impersonating our User-Agent cannot produce a valid signature.

Every request carries the Signature, Signature-Input and Signature-Agent headers. Our public key is published in the key directory:

HeiLocalBot is registered with Cloudflare as a Signed Agent, so our traffic is verified automatically on Cloudflare-protected sites.

How to control HeiLocalBot's access

HeiLocalBot follows the robots.txt standard. You can allow or block access, fully or on specific paths, by adding rules for the HeiLocalBot token in the robots.txt file at the root of your site:

# Block HeiLocalBot entirely
User-agent: HeiLocalBot
Disallow: /

# OR: allow everything, but exclude a folder and request a delay between requests
User-agent: HeiLocalBot
Disallow: /cart/
Crawl-delay: 5

Changes to robots.txt are picked up on the next visit. If you want your products to appear in Hei!Local, no action is needed — access is allowed by default.

Responsible crawling

• Respects robots.txt directives and Crawl-delay.
• Limits the number of requests per domain and keeps a delay between them.
• Applies adaptive backoff on 429 / 503 errors and honours the Retry-After header.
• Uses Last-Modified headers to avoid re-downloading unchanged content.
• Does not submit forms, does not authenticate, and does not access protected areas of the site.

Contact

For questions about HeiLocalBot: hei@heilocal.ro.