So, as per images below, when you search for an Australian community associated with lemmy, lemmy.world is more likely to come up than Aussie Zone in all i've tried, bar Melbourne our most active community.
My question: Is this a problem we should consider intentional action to correct? And if so what could we do?
I wonder if claiming ownership in Google Search Console would help? It would probably flow onto other search engines.
Though we might get ranked down for effectively duplicating a larger more commonly accessed site (Lemmy.world) even if the content belongs here. This could be a thing for the Lemmy developers to look into, i.e. generating a robots.txt which blocks all federated communities
~~Should just be able to do it with one rule of
Disallow: /c/*@*, I think?~~I forgot that federated posts are just under
/post/*like local ones are. You'd have to block each federated post individually. Not sure if you can do that on each individual page? Certainly adding every federated post to the top-level robots.txt isn't feasible.Posts have a canonical reference to the originating instance, e.g. this post contains
<link data-inferno-helmet="true" rel="canonical" href="https://aussie.zone/post/11962005">for me. This is a hint for search engines to ignore this post and instead index the original one instead. The same also already works for communities, this community containing<link data-inferno-helmet="true" rel="canonical" href="https://aussie.zone/c/meta">. Not sure if DDG is just ignoring this or there's another reason for it to show up multiple times.Yeah they might have to change how Lemmy handles posts like that which might break stuff. Search engines would also have to work out that Lemmy is not a single site but a collection and since the site name does not mention Lemmy it might be more difficult
Unsurprisingly i know next to nothing about Google Search Console. Looks like it could help though. At the very least you've given me an avenue to explore and build my own knowledge. Thank you!
Yeah it's good for working out why pages aren't being indexed. I'm not aware of anything similar for bing search