What our crawler does
StorePrism is the crawler behind StorePrism. It reads public pages from Shopify storefronts to work out what a store runs and what it sells. If you would rather it did not read yours, one line of robots.txt stops it — and this page tells you exactly what that line is, how soon it takes effect, and who to email if you would rather talk to a person.
How to identify us
Every request we make carries this header:
User-Agent: StorePrism/1.0 (+https://storeprism.com/bot)Requests come from our own servers. We do not use a residential proxy pool, we do not run JavaScript on your storefront, and we never sign in, add to a cart or place an order.
What we request, and how often
Three paths. There is no fourth, and you can check that against your access log.
| Path | Why | How often |
|---|---|---|
/robots.txt | To find out whether you want us to read the other two at all. | Once a day per store, before anything else. |
/products.json | The public product list Shopify storefronts publish. It is how we track what a store sells and how stock moves over time. | Between once an hour and once a week, depending on how much interest a store gets. Most stores are at the slow end. |
/ | The homepage, read once to identify what a store runs — its platform, its theme and the apps it loads. | When somebody looks the store up on one of our tools, and no more than once a day even then. |
We ask for one page at a time and space our requests out per host, so we should never be a noticeable share of your traffic. If we are, please tell us — that is a bug on our side and we would like to know.
How to stop us
Put this in the robots.txt at the root of your store:
User-agent: StorePrism
Disallow: /StorePrism is the only name we have ever crawled under, so that one line is all you need. If we ever rename, we will keep honouring every name we have ever published, permanently — a rename adds a name to that stanza and never takes one off it, because a rule you wrote years ago should not stop working because we changed our branding.
You can also block one path rather than all of them. If you are happy for us to say what platform your store runs on but would rather we did not read your product list, disallow just that:
User-agent: StorePrism
Disallow: /products.jsonWhat happens next
We re-read your robots.txt once a day, so a line added now takes effect within a day. After that we stop requesting the disallowed path entirely — we do not keep trying and we do not check whether you meant it.
If you take the line out again, the next time anyone looks your store up on one of our tools we will notice and start reading it again. If you would rather that happened sooner, email us.
We keep the readings we already took, which we think is the honest default: they were taken while you permitted them, and a store's history is not something we can reconstruct if you change your mind back. If you would like it removed as well, ask us and we will remove it.
Talking to a person
[email protected] reaches a human. Write to us if we are reading something we should not be, if we are asking too often, if you want your store removed, or if you would like us to start reading it again after a block. You do not need an account and you do not need to be a customer.
If your question is about a revenue figure we publish about your store, a correction to it, or the data we hold about an account, contact says which address answers what and how long we take.
One thing worth saying out loud
This site publishes its own robots.txt, and we expect crawlers to honour it. It took us longer than it should have to hold ourselves to the same rule — we were asking to be read on our terms while reading others on ours. This page and the behaviour behind it are that being fixed.