Bot policy

What our bot does, what it doesn't, and why.

ScrapsKitchen-ReferralBot/1.0

WHY THIS BOT EXISTS

When I started to build Scraps, I was building something for myself: a tool I could use to make sense of the odds and ends in my pantry, so that I could stave off another grocery store trip until the weekend.

Maybe I had a half-can of chipotles in adobo from a burrito recipe I got from a blog, an extra block of feta from last week's NYT Cooking recipe, and a pantry with some beans and tomatoes. This is that "I'm beyond the help of a recipe and I need to feed my family" situation. Once I had the app up and running, I just... kept building.

I think generative AI can be really useful. I think it's going to be a big part of our future (hopefully in more good ways than bad). But I also think there's a really stinky, unresolved ethical issue with how these LLMs came to be: big companies sucked up all the world's creative juices—that enchilada blog, NYT Cooking, church lady cookbooks—and have been using it to cannibalize our attention, to their benefit and at the expense of creators.

I want to keep building and maintaining scraps.kitchen. But I don't think there's an ethical way to do it if I'm not trying to point people back toward the cooks and writers who did all the real thinking.

That is what this bot is about. It does not scrape your site to train models. It only indexes your ingredient lists so that when our app generates a recipe using chipotles and feta, we can send our users directly to your site to see how the pros do it.

THE BOUNDARY

What we index vs. what we never touch.

What we index

  • ● Recipe title
  • ● Source URL
  • ● Author / publisher name
  • ● Ingredient list
  • ● Prep time

What we never touch

  • ● Step-by-step instructions
  • ● Blog narratives or article text
  • ● Photos or media
  • ● Any data used for model training

We read only the Schema.org/Recipe structured data (JSON-LD) that your site already publishes for search engines. We never parse the visual page, run JavaScript, or use a headless browser.

HOW IT WORKS

We read your ingredient list, check it against what our users have in their kitchens, and send them to your site.

01

We index your ingredient list

The bot reads the JSON-LD on your recipe pages and stores the title, author, ingredients, and a link back to you. That's it.

02

We match against real pantries

When a Scraps user generates an AI recipe, we check our index for human-crafted recipes with similar ingredients.

03

We send users to your site

If there's a match, we show your recipe as a "pro recipe match" and link directly to your page.

YOUR CHOICE

Opt out or opt in. It's your site.

Block the bot

Add this to your robots.txt and we'll stay away:

# Block the Scraps referral bot
User-agent: ScrapsKitchen-ReferralBot
Disallow: /

Already blocked us but want the referral traffic?

If your WAF or firewall blocked our bot and you'd like to be included in our referral index, reach out:

[email protected]

Curious why this exists?

Scraps started as one dad's attempt to stop wasting groceries. Here's the whole thing.

About Scraps