Asklemmy

This is actually my own Oh Shit story.

Early days of being a sysadmin and making changes on a major Linux server that we have in production. Running routine commands and changing permissions on a few directories and I make a typo. "sudo chmod 777 /etc/" instead of typing the rest of the directory tree I accidentally hit return.

It only ran for a fraction of a second before I hit CTRL + C to stop it but by then the damage had been done. I spent hours mirroring and fixing permissions by hand using a duplicate physical server. As a precaution we moved all production services off this machine and it was a good thing too as when we rebooted the server a few weeks later, it never booted again.

For those that don't know, chmod is used to set access permissions on files and folders, the 777 stands for "Read + Write + Execute" for the owner, group, and everyone else. The /etc directory contains many of the basic system configuration files for the entire operating system and many things have very strict permissions for security reasons. Without certain permissions in place those systems will refuse to load files or boot if not properly set.