https://lemmy.ml/robots.txt , https://lemmy.world/robots.txt , etc don't seem to disallow posts, so the text-based content should be easy to index, at least for these instances.
related news: Google is getting a lot worse because of the Reddit blackouts.
How long does it usually take for google to index websites? Because I tried the string lemmy site:lemmy.ml after:2023-06-15 and only one post turned up for me and it was Memes... the current state of affairs does not seem promising 😔 And if I tried with another instance with the same keywords lemmy site:kbin.social after:2023-06-15 nothing even turned up.
I wonder though, will search engines adapt to Lemmy and its fediverse system? Or will search engines die? Or will we see dedicated search engines to search through the fediverse?
How long does it usually take for google to index websites?
Anything between a couple of hours to more than a week, I don't think having a "real-time feed" through Google is important though. Other than world cup scores, their results were never about speed.
In my opinion, pushing lemmy to be a Reddit replacement isn’t the right move. Instead, Lemmy should be pushed as something that previously bulletin board forums dominated: the hobbyist, niche, discussion forum.
We don’t need hundreds of millions of users. Hell, most people stopped posting on places on Reddit that had too many users. What you want are communities with thousands of passionate users.
My point is that Google indexing would also be great, but more important is the ability to search these communicators for archivable posts. Reddit’s search functionality was dogshit so we had to rely on Google.
“communicators” = “communities”. I can’t seem to find an edit button on Mlem
Google brings it up
Oh, good point. Yes, probably? We can not simply assume search engines know that all of these point to the same content:
Or even worse, due to defederation, they may not all point to the exact same content.
Without further investment either from lemmy or the search engine's side, they are probably seen as distinct sources, not aggregated. Which makes each individually less relevant and less likely to show up .
Also note none of the adresses above contain 'lemmy'. How would users search for content on lemmy in these cases? Can't do "technology site:lemmy", or?
But I can say, lemmy content is visible. Haven't seen it on the first page of ecosia yet, but on page 2 or 3.
Maybe you could use use site:lemmy.ml, because they federate with most instances, they're likely to have most of lemmy's content?
A loosely moderated place to ask open-ended questions
If your post meets the following criteria, it's welcome here!
Looking for support?
Looking for a community?
~Icon~ ~by~ ~@Double_[email protected]~