this post was submitted on 01 Jul 2023
3725 points (97.1% liked)

Lemmy.World Announcements

29207 readers
283 users here now

This Community is intended for posts about the Lemmy.world server by the admins.

Follow us for server news 🐘

Outages πŸ”₯

https://status.lemmy.world/

For support with issues at Lemmy.world, go to the Lemmy.world Support community.

Support e-mail

Any support requests are best sent to [email protected] e-mail.

Report contact

Donations πŸ’—

If you would like to make a donation to support the cost of running this platform, please do so at the following donation URLs.

If you can, please use / switch to Ko-Fi, it has the lowest fees for us

Ko-Fi (Donate)

Bunq (Donate)

Open Collective backers and sponsors

Patreon

Join the team

founded 2 years ago
MODERATORS
 

Looks like it works.

Edit still see some performance issues. Needs more troubleshooting

Update: Registrations re-opened We encountered a bug where people could not log in, see https://github.com/LemmyNet/lemmy/issues/3422#issuecomment-1616112264 . As a workaround we opened registrations.

Thanks

First of all, I would like to thank the Lemmy.world team and the 2 admins of other servers @[email protected] and @[email protected] for their help! We did some thorough troubleshooting to get this working!

The upgrade

The upgrade itself isn't too hard. Create a backup, and then change the image names in the docker-compose.yml and restart.

But, like the first 2 tries, after a few minutes the site started getting slow until it stopped responding. Then the troubleshooting started.

The solutions

What I had noticed previously, is that the lemmy container could reach around 1500% CPU usage, above that the site got slow. Which is weird, because the server has 64 threads, so 6400% should be the max. So we tried what @[email protected] had suggested before: we created extra lemmy containers to spread the load. (And extra lemmy-ui containers). And used nginx to load balance between them.

Et voilΓ . That seems to work.

Also, as suggested by him, we start the lemmy containers with the scheduler disabled, and have 1 extra lemmy running with the scheduler enabled, unused for other stuff.

There will be room for improvement, and probably new bugs, but we're very happy lemmy.world is now at 0.18.1-rc. This fixes a lot of bugs.

(page 9) 50 comments
sorted by: hot top controversial new old
[–] [email protected] 1 points 2 years ago
[–] [email protected] 1 points 2 years ago

Thank you so much for doing this! The having an instance this big really made the difference for leaving reddit. I really missed jerboa and am glad to have it back as a client.

[–] [email protected] 1 points 2 years ago

On a somewhat related note, how are you coping with the scaling from the Reddit migration?

[–] [email protected] 1 points 2 years ago

Amazing work! Kids first, internet things later!

[–] [email protected] 1 points 2 years ago

Posting this from Jerboa. I am grateful for the hard work!

[–] [email protected] 1 points 2 years ago

Thanks for being both informative and open about this issue. Hope it works out.

[–] [email protected] 1 points 2 years ago

The speed and responsiveness of just scrolling is already infinitely better. Thanks :)!!!

[–] [email protected] 1 points 2 years ago

Tried to comment "looking good" but can't seem to comment. Getting a timeout.

[–] [email protected] 1 points 2 years ago

Thanks a lot, you’re great

[–] [email protected] 1 points 2 years ago

Nice, hopefully this'll fix the auto refreshing new posts on PC! Thanks man ^^

[–] [email protected] 1 points 2 years ago (2 children)

Question @[email protected] why update to the release candidate? Just want to help testing? Or was there some readdition (ie: captcha) that had you quick on the trigger?

load more comments (2 replies)
[–] [email protected] 1 points 2 years ago
[–] [email protected] 1 points 2 years ago

@ruud Thank you! This is an important one for the community! And maybe the servers too. ;)

[–] [email protected] 1 points 2 years ago

Thanks very much for your time and effort Ruud, it's much appreciated! Now, after you've put the kids to bed, grab yourself a beer and put your feet up!

[–] [email protected] 1 points 2 years ago

Yay, thank you!

[–] [email protected] 1 points 2 years ago

Thank you for the work and all the transparency Ruud, glad to have you as our Admin.

[–] [email protected] 1 points 2 years ago

Thanks for your work!

[–] [email protected] 1 points 2 years ago

Good job! πŸŽ‰πŸŽ‰

[–] [email protected] 1 points 2 years ago

Great work, thanks!

[–] [email protected] 1 points 2 years ago

Seems to be working great so far.

[–] [email protected] 1 points 2 years ago

Thank you! Your hard work is truly appreciated.

This whole upgrade process reminded me to check the patreon donation stuff for y'all.

[–] [email protected] 1 points 2 years ago

Congratulations on getting it up and running!

[–] [email protected] 1 points 2 years ago

Congratulations on getting it up and running!

[–] [email protected] 1 points 2 years ago

Well done guys. Love the new update

load more comments
view more: β€Ή prev next β€Ί