this post was submitted on 15 Jul 2023

261 points (99.6% liked)

Fediverse

28278 readers

1388 users here now

A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, KBin, etc).

If you wanted to get help with moderating your own community then head over to [email protected]!

Rules

Posts must be on topic.
Be respectful of others.
Cite the sources used for graphs and other statistics.
Follow the general Lemmy.world rules.

Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration), Search Lemmy

founded 1 year ago

MODERATORS

[email protected]

261

Search Lemmy 0.4.0 update (an enhanced search engine for Lemmy) (www.search-lemmy.com)

submitted 1 year ago by [email protected] to c/[email protected]

34 comments fedilink hide all child comments

A couple days ago I updated https://search-lemmy.com/ to 0.4.0.

New features, that several people were asking for:

The UI has been overhauled and it should be much easier to find your home instance now.
Search itself has been overhauled. Increase search performance significantly. I also automatically search for related terms as well. You may now see fewer search results, but ideally they should be more relevant. You can also now include basic syntax like:
- quotes: "some terms that must be together"
- negative terms: cat -dog (shows posts about cats that don't mention dogs)
- either or: cat OR dog (shows posts about either cats or dogs). The default search behavior is now an implicit AND, but order doesn't matter.
I've added several new filters that you can use including:
- !safeoff -- Disables safe search allowing NSFW posts to appear in the search results (NSFW is now hidden by default)
- since:YYYY-MM-DD -- shows only posts that have occurred since the specified date
- until:YYYY-MM-DD -- same as above but in reverse. It will only posts up to the given date.
I've removed the preferred-instance query parameter from the results URL so it should be easier to share links to search results now.
The date the post was created or last updated is now displayed in the search results.

Bug Fixes:

Site performance should now be stable. Fixed a bug related to the database pool that was causing the site to hang.
Fixed a bug that would cause broken links.
Fixed various bugs with the crawler causing posts to be missed.

Known Issues:

If you set your home-instance to a fairly small instance, the number of search results is also relatively small. Once (https://github.com/LemmyNet/lemmy/issues/3259) is resolved. I should be able to show links regardless of what your home instance is set to, allowing you to search the entire Fediverse.
Currently searching only looks at the post title and body. Comments aren't indexed either. This also is dependent on the above issue on Lemmy itself.

Finally some things to note:

I've started to refactor the code to abstract away Lemmy from the actual search engine. As I now start to prepare to search other Fediverse instances like Kbin, and maybe even Mastodon, etc...

top 30 comments

sorted by: hot top controversial new old

[–] [email protected] 19 points 1 year ago (1 children)

I've saved this, but it would be nice to see the syntax somewhere in the search engine

[–] [email protected] 2 points 1 year ago

Look under search tips for the filters at least.

[–] [email protected] 7 points 1 year ago (2 children)

What about it I want to find the most populated community for a given search term, across the lemmiverse?

[–] [email protected] 4 points 1 year ago (1 children)

Currently you can just search for posts. I don't track anything like the number of members in a community etc.. just the content of the post and how more or less accurate they are to your current query. I'm continuously trying to improve the page rankings though.

I guess in theory you can perform the same search multiple times with different community:!some_community@some_instance filters to see which returns the most results, but ya, that wouldn't be the most convenient. At the moment this tool though is about finding posts, but who knows what features I may add in the future.

[–] [email protected] 1 points 1 year ago (1 children)

Sounds good, figured it couldn’t hurt to ask

[–] [email protected] 3 points 1 year ago

I've been seeing a few people ask for something like this recently, so I might try and see how hard it would be to build something to help find active communities.

[–] [email protected] 4 points 1 year ago (1 children)

I think you should use Lemmy explorer for this.

[–] [email protected] 2 points 1 year ago* (last edited 1 year ago)

Thanks, good calln

[–] [email protected] 3 points 1 year ago* (last edited 1 year ago) (1 children)

Thank you for making this!

I don’t get any results no matter what I search for. The page refreshes, but no results (or errors) appear.

Turning my adblockers off had no effect. Same behaviour in Safari and Firefox on iPadOS 16.5.x.

Edit: Oops, I didn’t select an instance and the default instance (Ice Orchid) returned no results. Changing to my home instance fixed it. Maybe default should get changed to Lemmy.world or something big?

[–] [email protected] 1 points 1 year ago (1 children)

Thanks. Adding an issue for that. I should be able to set the default instance to the 'seed-instance' that's configured for the crawler.

[–] [email protected] 1 points 1 year ago (1 children)

Cheers!

[–] [email protected] 1 points 1 year ago (1 children)

https://github.com/marsara9/lemmy-search/issues/45. If you want to track it.

[–] [email protected] 1 points 1 year ago

Thanks very much, I appreciate that!

[–] [email protected] 2 points 1 year ago

Great, thank you for your work!

[–] [email protected] 2 points 1 year ago

GOAT behaviour, I'll bookmark it!

[–] [email protected] 1 points 1 year ago (2 children)

I created [email protected] -- searching for "ultralight" returns zero results on your site https://search-lemmy.com/results?query=ultralight&page=1 whereas searching for it at least returns something on each lemmy instance i've searched

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago) (1 children)

some query community:[[email protected]](/c/[email protected]) would search your community. Now, if this community is less than 24hrs old, it may not have been indexed yet, so you may just need to wait a day or so.

I just tested with https://www.search-lemmy.com/results?query=test+community%3A%21ultralight%40lemmy.world&page=1 I was able to at least find 2 posts.

[–] [email protected] 1 points 1 year ago (2 children)

very cool, thanks for the feedback and the site! my question is more along the lines of whether someone interested in a topic would be able to find my community without knowing it existed in the first place?

[–] [email protected] 2 points 1 year ago (1 children)

Just added a new feature that lets you search but it returns the number of matches per community. So you should be able to use that to find the most active communities based on your search result.

[–] [email protected] 1 points 1 year ago (1 children)

Cool, how do I do it? https://www.search-lemmy.com/results?query=ultralight still returns "Found 0 results in 0.01 seconds"

[–] [email protected] 1 points 1 year ago (1 children)

https://www.search-lemmy.com/find-communities/results?query=camping&page=1

It's under the "Find Communities" button at the top of the screen. If you don't see that button, try clearing your browser cache.

[–] [email protected] 1 points 1 year ago (1 children)

cleared cache, your url still gives me no results. in the top right the dropdown as "Ice Orchid". I'm not sure what that is

[–] [email protected] 2 points 1 year ago (1 children)

Here's the landing page if you just go to https://www.search-lemmy.com/. I'm assuming that drop-down that you're referring to is your home instance selector. Since you're on lemmy.world I suggest you set that to well, lemmy.world. Then you can do your search and all of the results will take you directly to that post on lemmy.world (or whatever you set as your home instance).

Now you can also see that Find Communities button in the top right, you can click on it and it'll take you to a similar page but instead of returning posts for search results, it will return a list of communities, based on how many matches it found. (as if you did a search on the normal page but instead just counted the number of results per community).

[–] [email protected] 1 points 1 year ago (1 children)

why make users set a home instance? i can see a list of communities local to lemmy.world at https://lemmy.world/communities in the [Local] tab, and i can search my home instance at https://lemmy.world/search. i figured a search engine would span instances so users could find things across the lemmy-verse?

[–] [email protected] 2 points 1 year ago* (last edited 1 year ago)

Eventually it will. But there's a bug preventing global search working in lemmy itself. You can see more of the details here (https://github.com/marsara9/lemmy-search/issues/20 )

One of my primary goals with this is that users MUST be able to open a given link in their home instance so that they can then interact / reply / subscribe, etc... without having to figure out how to find said post themselves. So with that requirement, users MUST select a home instance but because of the before-mentioned bug I can't show posts that your instance isn't aware of.

[–] [email protected] 1 points 1 year ago

depends on what they search for, for example this finds several posts on your community:

https://www.search-lemmy.com/results?query=lightweight+backpacking&page=1

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago)

Replying from my lemmy.ml account since lemmy.world appears to be down but:

I just tested with https://www.search-lemmy.com/results?query=test+community%3A%21ultralight%40lemmy.world&page=1 I was able to at least find 2 posts.

[–] [email protected] 1 points 1 year ago

Thank you.

[–] [email protected] 1 points 1 year ago (1 children)

Do these changes automatically carry over to 3rd party apps?

[–] [email protected] 3 points 1 year ago

This doesn't change the behavior of the built-in search within Lemmy. But rather this is suppose to be a close approximation of using Google with adding reddit to the end of your query.

The problem with the fediverse is that there are so many different instances you can't really include them all in a search query and even if you could the links that Google would provide wouldn't necessarily go to YOUR instance. This aims to fix that.

load more comments