The short version
Shelfbee — a sole proprietorship registered in Ontario — compares grocery and household prices across Canadian retailers. To do that, we read publicly visible product and price information from retailer websites and digital flyers. We do this politely, sparingly, and within the spirit of each retailer's public terms.
What we collect
For each item we track, we record:
- The product name shown on the retailer's search result
- The advertised price and pack size
- Whether the item is on sale and the regular price if shown
- The product URL on the retailer's site
- The image URL the retailer publishes for the product (we link to it; we do not rehost it)
- The store and approximate location used for the listing
We do not collect, parse, or use any personal account or session data from any retailer. We do not bypass paywalls or login walls. We do not crawl entire catalogues — every URL we visit is one we already know is public.
How often we visit
- Weekly canonical scrape — every Friday morning, we update our tracked product list across the retailers we directly support. This is the bulk of our requests.
- On-demand searches — when a user searches for something we haven't tracked yet, we may look it up at one or two retailer sites in real time, then cache the result for 30 minutes so the same query doesn't hit the retailer again.
Our rate limits
- Maximum 1 request every 3 seconds per retailer domain
- Single-threaded — we do not run parallel requests against any one retailer
- We respect each retailer's robots.txt
- We back off and skip URLs that return CAPTCHA or anti-bot challenges; we do not solve them
How to identify our requests
Our crawler uses the user-agent string:
ShelfbeeBot/1.0 (+https://shelfbee.ca/about/scraper)
On retailer sites with strong bot protection, we may use a standard browser user-agent because bots are blocked outright. In those cases we still adhere to the rate limits above.
Retailer requests
If you operate a retailer website and would prefer Shelfbee not include your listings, email scraper@shelfbee.ca and we'll remove your domain from our crawl list within 24 hours. We'd rather work with you than around you — if there's a structured feed or partnership option that suits both sides better, we're open to that conversation.
Compliance with Canadian law
Shelfbee operates from Toronto and stores all user data in Canada. Our scraping activity follows the Canadian Competition Act's rules on accurate advertising — we never claim "cheapest in Toronto" or similar absolutes. Our results show the lowest tracked price among the stores we cover, with the time of last verification displayed on every offer.
Contact
Press / partnerships: hello@shelfbee.ca. Crawler concerns: scraper@shelfbee.ca.