Hi, I’m building a personal website and I don’t want it to be used to train AI. In my robots.txt file I blocked:

  • ChatGPT-User
  • GPTBot
  • Google-Extended
  • FacebookBot

What bots should I also add? Are there any other ways to block AI bots?

IMPORTANT: I don’t want to block search engine crawlers, only bots that are used to train AI.

  • wagoner@infosec.pub
    1·
    1 year ago

    So that leaves two options then. Leave the front door wide open, don’t bother with any locks. Or shut down the web site. I’m for at least closing the door with the right robots.txt