I'm always amazed how after just a few weeks my fitness levels go way down.
It's well annoying, innit? You don't get that with muscle building — take a couple of weeks or so off of that and you may even come back stronger. But fitness? It's bloody relentless how you need to keep on top of it.