I think there are a number of thigs people generally don't allow kids to have/do, not because it's immoral but because it creates a hassle for you, the parent. Take sharpies or other permanent markers for example. If you give one to a 4 or 5 year old, not matter how much you teach them about "how and when" to use them, you're still guaranteed to get sharpie marks on places you don't want them, and while it's not the end of the world it can be a drag.
Of course it's up to you to decide when's the appropriate age to have access to permanent markers, cans of spray paint and pocket knives.
The issue with swearing is that kids copy everything you do, so even before school age you can create hassles if your four year old shouts out at the wrong moment "Eat shit, ya fuckin cunt!" This might sound funny but it's possible to unintentionally set your kids up with habitual patterns of behavior that create unnecessary drama -both for you and your child- that would have been a lot easier to avoid in the first place.