I want to build an extremely robust and scalable web server using nginx and MySQL as the database behind it. No I don't need to, this would just be for fun and a learning experience.
The problem is I really don't know where to start with a proper base plan to this.
I own two houses in neighboring towns with gigabit Internet at each location. I have unifi networking hardware at both locations with a VPN enabled between.
Currently no servers at the one location and the VPN is used for cameras routing back to the other location.
I have a stack of HP mini PCs of which I currently have a few running XCP-NG with a storage cluster across and running just a few base servers such as domain controllers and a few Linux servers hosting a handful of applications that we use around the house.
I have a half dozen more identical machines at my disposal but I don't know that xcp is the proper hypervisor for this project in the first place.
I have never done anything truly highly available or multi datacenter before.
Since this is for fun and not because I need to, there is not a huge budget but I do have a little money to play with that I would like to use in the best way.
I am flexible with replacing firewalls if there is something that will get me closer to my goal.
I am ok with some licensing, preferably one time cost.
Any thoughts? Or is this something that's outside of reach of a home lab budget?
Edit: I do have a Synology with quite a bit of storage but that is dedicated to backups and will not be used to store any live data.