this post was submitted on 28 Feb 2024
341 points (98.0% liked)
Technology
60101 readers
1871 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Can we get a list of companies NOT doing this? I'd assume it's going to be much shorter.
All these AI and machine learning companies are taking content directly from websites and ignoring robot.txt files.
If your content is able to be crawled, even without being listed on search engines, I don't think it really matters.
It might help proof an AI company against legal issues that might be brought about by their using the content. If they're ever sued by Automattic, then they can just point to the deal and say that they bought the data from them. There's much less ambiguity.
You are correct, about the legal stuff. These companies are being sued all the time.
Doing this deal also makes processing the data a lot easier. Being handed a big ass database would be a lot easier than crawling for content.
What I posted was about how they operate. These companies showed time and time again that they don't really care what data they are taking or from whom. They will even take their own AI or machine learning content and put it in their own system.