I hate to bear the bad news, but as long as the model is too large to fit entirely in VRAM getting 5 t/s on a 8x7b is going to be difficult. You can throw another 16gb RAM in the system which could help with caching and context length, but since the model is still having to juggle data in and out of VRAM the speeds will remain low.

I wouldn't upgrade the CPU personally, focus on adding beefier GPU. And it's probably not worth adding the 970 to the mix, the 4GB isn't providing much room and will likely slow down the 6800 XT more.

Ollama doesn't currently support mixing CUDA & ROCm. https://github.com/ollama/ollama/issues/3723#issuecomment-2071134571

One thing to keep in mind about adding RAM your speed could drop depending on how many slots you populate. For me, I have a 5700G and with 2x16Gb, it runs at 3200Mhz, but with 4x16Gb(same exact product), it only runs at 1800Mhz. In my case, RAM speed has a huge effect on tokens/sec, if I have a model that has to use some RAM.

You can check AMD's spec page for your processor, but they don't really document a lot of this stuff.

I don't know how important the CPU is for those workloads tbh. but I feel like not as important so maybe you're fine leaving it as it is.

I think AMD wanted to release a new GPU lineup (radeon 8000 series) sometime this year/early next year. Maybe just wait for that, sell your old card on the used market and buy a new one?

(And throw in 16G of RAM as you said)