Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
jorams
on Jan 16, 2025
|
parent
|
context
|
favorite
| on:
Nepenthes is a tarpit to catch AI web crawlers
Yes, all the big search engines respect that meta tag. Some of the big abusive AI crawlers do too, kind of defeating the (stated) point of the tarpit.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search: