About PantriBot

Why PantriBot visits

Every visit begins with a person. Either they searched for a recipe by name or by ingredient, or they opened their recipe suggestions in Pantri, which asks for real pages for a small number of dishes that match what they have at home.

A search provider suggests candidate pages; PantriBot retrieves a bounded set of them to inspect the structured recipe data they publish. Per open of the suggestions, PantriBot looks at no more than six dishes and fetches no more than three candidate pages for each, each page once per visit. A dish Pantri has already found is served from Pantri's own copy without visiting you again. The person did not necessarily name your site.

What it reads and shows

PantriBot looks for structured Recipe data already published in the page, such as the recipe name, image, cooking time, yield, ingredients, instructions and nutrition. Pantri shows those fields inside the app so the person can decide what to cook, and may reformat the instructions into step-by-step cooking mode.

Every recipe keeps a visible, clickable link to its original publisher, on the suggestion card and on the recipe screen. The recipe image is loaded by the person's device directly from your server when the recipe is on screen; Pantri's servers do not copy it. Pages explicitly marked isAccessibleForFree: false in their structured data are skipped.

What Pantri keeps

The recipe fields PantriBot read for a dish are kept on Pantri's servers so the next person who asks for that dish gets them without a new visit to your site. These copies are served only to the Pantri app; they are not offered as a public website, a public database or a search API. Pantri keeps a copy for as long as it keeps suggesting that dish.

A kept copy is refreshed or removed when your robots.txt stops allowing PantriBot (checked at least every 24 hours before it is served), when the page can no longer be reached, or when you ask.

What it does not do

• ✓It does not run a background or scheduled crawl.
• ✓It does not follow links on a page to discover more pages.
• ✓Pantri does not use fetched recipe content to train machine-learning models.
• ✓It does not build a public search index or a public recipe database. What it keeps is described above and reaches only the Pantri app.
• ✓It does not attempt to bypass a login, paywall or bot-protection challenge.

How to control access

PantriBot checks /robots.txt before every page fetch. A specific PantriBot group takes priority; if none exists, it follows the rules for User-agent: *.

To refuse PantriBot entirely, add:

User-agent: PantriBot
Disallow: /

To allow one section while refusing everything else:

User-agent: PantriBot
Disallow: /
Allow: /recipes/

A rule you add later also applies to copies Pantri already kept: they stop being served within 24 hours of PantriBot seeing the new rule.

Rules written only for other named agents — such as GPTBot, ClaudeBot or CCBot — do not apply to PantriBot. PantriBot does not train models or conduct an open-ended crawl, so it identifies and evaluates itself separately. If you would rather it stayed out, the two lines above are all it takes.

Robots policy details

• ·Valid policies are normally refreshed within 24 hours.
• ·An HTTP 4xx response for /robots.txt is treated as unavailable rules, in line with RFC 9309, so the fetch may proceed.
• ·A network error or server error fails closed when no earlier valid policy exists. A recently cached valid policy may be used briefly if refreshing it fails.
• ·Published Crawl-delay values are honoured. If the requested delay does not fit within the page-fetch budget, the source is skipped.

Verifying requests

User-agent strings can be copied by anyone. Pantri does not currently publish a verified outbound IP list or reverse-DNS scheme, so the user-agent by itself is not proof that a request came from Pantri. This page will be updated if a verification method is introduced.