Log Collection (lemmy.procrastinati.org)

submitted 1 year ago by [email protected] to c/[email protected]

13 comments fedilink hide all child comments

I'm in the process of re-configuring my home lab and would like to get some help figuring out log collection. My setup was a hodgepodge of systems/OSes using rsyslog to send syslogs to a syslog listener on my qnap but that's not going to work anymore (partly because the qnap is gone).

My end-goal is going to be as homogeneous as I can manage. Mostly Debian 12 systems (phy and vm) and Docker containers. Does anyone know of a FOSS solution that can ingest journald, syslog, and if it's even possible to send docker logs to a log collector?

Thanks

you are viewing a single comment's thread
view the rest of the comments

[-] [email protected] 7 points 1 year ago

You could use grafana loki to handle logs, it's similar to Prometheus so if you're already using that and/or grafana it's an easy setup and the API is really simple too.

[-] [email protected] 3 points 1 year ago

I second this. Loki for logs, VictoriaMetrics for metrics―it's significantly more lightweight than ELK logging (and any lags are irrelevant for a homelab), and VM is similarly much more careful with RAM than Prometheus.

[-] [email protected] 2 points 1 year ago

I did stumble on Grafana Loki in my search. Was trying to figure out if it was overkill. Is it fairly lightweight?

[-] [email protected] 2 points 1 year ago* (last edited 1 year ago)

Much less intensive than an elasticsearch anything. I have Loki, grafana and 3 promtail clients running for my env (switched from Graylog/elasticsearch) and over the last few days Loki is sitting at 3GB memory and 8% CPU processing logs for about 6 devices.

[-] [email protected] 1 points 1 year ago

Ok thanks. Looks like I can give it a try in docker.

this post was submitted on 25 Aug 2023

20 points (91.7% liked)

Selfhosted

39251 readers

254 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago

MODERATORS

[email protected]