Drstrangelove hit the nail on the head IMO. Bulk dechlorinators like Safe or bulk Sodium Thiosulphate are so cheap that is seems crazy to not use them, regardless of how little chlorine your water supply company uses on a good day.
Just remember that when people say their fish "seem fine" when not using dechlorinator they are usually basing that statement on nothing more than what they can see while watching their fish, not any sort of in-depth study of things like damage to gill tissue over a lifespan of many many years.