Home manager manages dot files, which aren’t managed at the Nixos config level. Sine this means managing the version of the corespondent package related to a dotfile config, home manager has to manage packages.
So, in my world: the only thing in home manager are programs which have dot files that need to be managed… everything else is in the Nixos config level. Except for local dev environments, which have flakes associated with them.