(Gonna expand on a comment I whipped out yesterday - feel free to read it for more context)
At this point, its already well known AI bros are crawling up everyone's ass and scraping whatever shit they can find - robots.txt, honesty and basic decency be damned.
The good news is that services have started popping up to actively cockblock AI bros' digital smash-and-grabs - Cloudflare made waves when they began offering blocking services for their customers, but Spawning AI's recently put out a beta for an auto-blocking service of their own called Kudurru.
(Sidenote: Pretty clever of them to call it Kudurru.)
I do feel like active anti-scraping measures could go somewhat further, though - the obvious route in my eyes would be to try to actively feed complete garbage to scrapers instead - whether by sticking a bunch of garbage on webpages to mislead scrapers or by trying to prompt inject the shit out of the AIs themselves.
The main advantage I can see is subtlety - it'll be obvious to AI corps if their scrapers are given a 403 Forbidden and told to fuck off, but the chance of them noticing that their scrapers are getting fed complete bullshit isn't that high - especially considering AI bros aren't the brightest bulbs in the shed.
Arguably, AI art generators are already getting sabotaged this way to a strong extent - Glaze and Nightshade aside, ChatGPT et al's slop-nami has provided a lot of opportunities for AI-generated garbage (text, music, art, etcetera) to get scraped and poison AI datasets in the process.
How effective this will be against the "summarise this shit for me" chatbots which inspired this high-length shitpost I'm not 100% sure, but between one proven case of prompt injection and AI's dogshit security record, I expect effectiveness will be pretty high.
How about honeypotting? What's the chance the crawlers are written smart enough to avoid a neverending HTTP stream?
So this is an idea from SSH: you make a server that listens at port 22 and responds to any connections with a valid, but extremely long message slowly fed to the source byte by byte. Automated bots that look for open SSH ports or vulns get trapped there, and they have to keep consuming resources to service the connection.
Also what happens if you try to feed it an infinite HTML file very quickly? Like just spam the stream with
<div><div><div>...?Given the security record I mentioned earlier, their generally indiscriminate scraping and that one time John Levine tripped up OpenAI's crawler, I suspect its pretty high.