655
submitted 1 month ago by [email protected] to c/[email protected]
top 50 comments
sorted by: hot top controversial new old
[-] [email protected] 149 points 1 month ago

Cleanup

Check current disk usage:

sudo journalctl --disk-usage

Use rotate function:

sudo journalctl --rotate

Or

Remove all logs and keep the last 2 days:

sudo journalctl --vacuum-time=2days

Or

Remove all logs and only keep the last 100MB:

sudo journalctl --vacuum-size=100M

How to read logs:

Follow specific log for a service:

sudo journalctl -fu SERVICE

Show extended log info and print the last lines of a service:

sudo journalctl -xeu SERVICE

[-] [email protected] 38 points 1 month ago

I mean yeah -fu stands for "follow unit" but its also a nice coincidence when it comes to debugging that particular service.

[-] [email protected] 4 points 1 month ago
[-] [email protected] 23 points 1 month ago

--vacuum-time=2days

this implies i keep an operating system installed for that long

[-] [email protected] 9 points 1 month ago

something something nix?

[-] [email protected] 19 points 1 month ago

sudo journalctl --disk-usage

panda@Panda:~$ sudo journalctl --disk-usage  
No journal files were found.  
Archived and active journals take up 0B in the file system.

hmmmmmm........

[-] [email protected] 12 points 1 month ago* (last edited 1 month ago)
user@u9310x-Slack:~$ sudo journalctl --disk-usage  
Password:  
sudo: journalctl: command not found  
[-] [email protected] 20 points 1 month ago* (last edited 1 month ago)

seems like someone doesn't like systemd :)

[-] [email protected] 5 points 1 month ago

I don't have any feelings towards particular init systems.

[-] [email protected] 4 points 1 month ago

Just curious, what distro do you use that systemd is not the default? (I at least you didn't change it after the fact if you don't have any feelings (towards unit systems ;) ) )

[-] [email protected] 5 points 1 month ago
[-] [email protected] 9 points 1 month ago

Badass! Thanks!

[-] [email protected] 8 points 1 month ago

Thank you for this, wise sage.

Your wisdom will be passed down the family line for generations about managing machine logs.

[-] [email protected] 5 points 1 month ago

Glad to help your family, share this wisdom with friends too ☝🏻😃

load more comments (1 replies)
[-] [email protected] 3 points 1 month ago

Actually something I never dug into. But does logrotate no longer work? I have a bunch of disk space these days so I would not notice large log files

load more comments (5 replies)
[-] [email protected] 3 points 1 month ago

If you use OpenRC you can just delete a couple files

load more comments (3 replies)
[-] [email protected] 31 points 1 month ago

Try 60GB of system logs after 15 minutes of use. My old laptop's wifi card worked just fine, but spammed the error log with some corrected error. Adding pci=noaer to grub config fixed it.

[-] [email protected] 24 points 1 month ago

I had an issue on my PC (assuming faulty graphics driver or bug after waking from sleep) that caused my syslog file to reach 500GiB. Yes, 500GiB.

[-] [email protected] 9 points 1 month ago* (last edited 1 month ago)

Nearly 700gb in logs

wtf 🤯

load more comments (1 replies)
[-] [email protected] 30 points 1 month ago* (last edited 1 month ago)
[-] [email protected] 5 points 1 month ago
[-] [email protected] 4 points 1 month ago

Ah, yes, the standard burger size.

[-] [email protected] 30 points 1 month ago

You just need a bigger drive. Don't delete anything

[-] [email protected] 10 points 1 month ago

Oh lord watch me hoard

[-] [email protected] 26 points 1 month ago* (last edited 1 month ago)

Once I had a mission critical service crash because the disk got full, turns out there was a typo on the logrotate config and as a result the logs were not being cleaned up at all.

edit: I should add that I used the commands shared in this post to free up space and bring the service back up

[-] [email protected] 25 points 1 month ago* (last edited 1 month ago)

Fucking blows my mind that journald broke what is essentially the default behavior of every distro's use of logrotate and no one bats an eye.

[-] [email protected] 25 points 1 month ago* (last edited 1 month ago)

I'm not sure if you're joking or not, but the behavior of journald is fairly dynamic and can be configured to an obnoxious degree, including compression and sealing.

By default, the size limit is 4GB:

SystemMaxUse= and RuntimeMaxUse= control how much disk space the journal may use up at most. SystemKeepFree= and RuntimeKeepFree= control how much disk space systemd-journald shall leave free for other uses. systemd-journald will respect both limits and use the smaller of the two values.

The first pair defaults to 10% and the second to 15% of the size of the respective file system, but each value is capped to 4G.

[-] [email protected] 13 points 1 month ago

If anything I tend to have the opposite problem: whoops I forgot to set up logrotate for this log file I set up 6 months ago and now my disk is completely full. Never happens for stuff that goes to journald.

[-] [email protected] 4 points 1 month ago* (last edited 1 month ago)

It can be, but the defaults are freaking stupid and often do not work.

[-] [email protected] 18 points 1 month ago

Aren’t the defaults set by your distro?

load more comments (1 replies)
[-] [email protected] 5 points 1 month ago

Still boggles my mind that systemd being terrible is still a debate. Like of all things, wouldn’t text logs make sense?

[-] [email protected] 5 points 1 month ago

Wait… it doesn’t store them in plaintext?

[-] [email protected] 4 points 1 month ago

Nope, you need journalctl to read.

load more comments (1 replies)
load more comments (1 replies)
[-] [email protected] 4 points 1 month ago

Wouldn't compressed logs make even more sense (they way they're now)?

[-] [email protected] 16 points 1 month ago

This once happened to me on my pi-hole. It's an old netbook with 250 GB HDD. Pi-hole stopped working and I checked the netbook. There was a 242 GB log file. :)

[-] [email protected] 11 points 1 month ago

Systems/Journald keeps 4GB of logs stored by default.

[-] [email protected] 10 points 1 month ago
[-] [email protected] 10 points 1 month ago

Recently had the jellyfin log directory take up 200GB, checked the forums and saw someone with the same problem but 1TB instead.

[-] [email protected] 16 points 1 month ago* (last edited 1 month ago)

2024-03-28 16:37:12:017 - Everythings fine

2024-03-28 16:37:12:016 - Everythings fine

2024-03-28 16:37:12:015 - Everythings fine

[-] [email protected] 7 points 1 month ago* (last edited 1 month ago)

logrotate is a thing.

[-] [email protected] 6 points 1 month ago

Windows isn't great by any means but I do like the way they have the Event Viewer layout sorted to my tastes.

[-] [email protected] 3 points 1 month ago

True that. Sure, I need to keep my non-professional home sysadmin skills sharp and enjoy getting good at these things, but I wouldn't mind a better GUI journal reader / configurator thing. KDE has a halfway decent log viewer.

It might also go a long way towards helping the less sysadmin-for-fun-inclined types troubleshoot.

Maybe there is one and I just haven't checked. XD

[-] [email protected] 6 points 1 month ago

I recently discovered the company I work for, has an S3 bucket with network flow logs of several TB. It contains all network activity if the past 8 years.

Not because we needed it. No, the lifecycle policy wasn't configured correctly.

[-] [email protected] 5 points 1 month ago

I couldn't tell for a solid minute if the title was telling me to clear the journal or not

load more comments
view more: next ›
this post was submitted on 01 Aug 2024
655 points (98.5% liked)

linuxmemes

20747 readers
362 users here now

I use Arch btw


Sister communities:

Community rules

  1. Follow the site-wide rules and code of conduct
  2. Be civil
  3. Post Linux-related content
  4. No recent reposts

Please report posts and comments that break these rules!

founded 1 year ago
MODERATORS