Yes the water is "safe" to drink but if you value your health, i would advise against doing so, especially if you have access to filtered and/or bottled water.
The municipal water supplies (and this includes the UK too) are loaded with environmental toxins and other poisons including fluoride and chlorine, as well as pesticides and other lovelies. If you get a water distiller and run a cycle, what is left at the bottom of the chamber is enough to turn your stomach. Ok so it's diluted in a gallon of water but even so.
Personally, filtered and/or bottled water is not only safer, but tastes so much better, and makes the food you use to cook with it or those cups of tea taste better too.
Even if i had my own water well i would filter it as you do not know what that water has come into contact with before it got to you.
Yes, if your dying of thirst then by all means knock yourself out. Otherwise, if you have the option to filter it, then do that as a default is my advice.