Generally in the west (US/Canada, and most of Europe) tap water is safe to drink. I've been to Iceland and don't remember tap water being a concern. This is something you should double check before every trip though. A good rule of thumb is just going by how developed/rich the country is that you are visiting, with more developed countries usually having potable tap water but this is not a guarantee. (And some countries are far too large and diverse to apply this rule efficiently)
Also asking the locals is not necessarily a good idea either as there immune system might be accustomed to the different bacteria and pollutants in the water. For example drinking tap water in some places in the Middle-East might not be an issue for the locals but as I haven't grown up there I probably wouldn't risk drinking tap if I can avoid it, not to ruin the trip with getting sick.
TLDR: check with your country's official travel recommendations
Edit: someone mentioned bottleded water just being bottled tap water. While this is not uncommon in Europe (not sure about the rest of the world), the water does go through extra steps of filtering and cleaning meaning it might be a bit safer to foreigners.
I agree but use docker-compose instead!