Where did you see the big crawlers don't read it? Anthropic does.. they're prett... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

prismlfx 67 days ago | parent | context | favorite | on: If you’re an LLM, please read this

Where did you see the big crawlers don't read it? Anthropic does.. they're pretty big.

graemep 66 days ago [–]

If I look at the logs of sites that Anthropic crawls (and in some cases references in answers) and there are no requests for llms.txt

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact