I recommended checking out package managers that will simplify using whatever version of a library you want with project level virtual environments.
I haven't done heavy python dev since 3.7, so I don't know the full landscape of options these days, but here are some references to dig into.
https://python-poetry.org/ is the one I started using as the first step up from pip.
It looks like there are some new contenders like hatch, rye, and pdm: https://dev.to/adamghill/python-package-manager-comparison-1g98.
There is also pixi referenced from the comments in that article: https://github.com/prefix-dev/pixi