Do you know how hard your water is? GH and KH? If not, what's your tank pH?
You really need to have a GH & KH test kit before trying to raise or lower the hardness and/or alkalinity of your water. Yes, tannic acids from peat and other substances will lower the hardness and the alkalinity, but in hard water, their not going to make a significant difference. Also, because water changes must be maintained, more tannins must be added at each water change. We're talking about a LOT of tannins here.
I know someone who had moderately hard water (pH was around 7.8 after aging 24 hours) who used to make around 5 gallons of 'peat tea' each week using a coffee maker (coffee urn I believe) and add that to his 75 gallon tank with each 75% weekly water change. I believe it lowered GH and KH about 4º each. TDS dropped around 150 ppm and pH dropped to 7.4.
You'd get the same drop in GH, KH and TDS as he did (4dKH, 4dGH, -150 tds) no matter how hard your water is if you mixed in the same ratio of 'peat tea' to water. But, with very hard water where your GH and KH may be 15º to 20º or higher, what's the point of going through all the trouble for 4º?
Tannins do have antifungal, antiparasitic, antibacterial and likely other beneficial properties which is a much better reason for adding them than trying soften your water imo. In some South American rivers tannins stain the water as dark as coffee. Fish have lived with these substances in the wild for millions of years, but tap water is completely void of them. This is why many fishkeepers .. especially those with SA softwater species ... add tannins to their tanks.
Tannins can also be made from nuts, cones, leaves and driftwood of hardwood trees (not evergreens). But, they MUST be picked from the ground in the Fall and NOT off the tree. (The tree removes substances you don't want in your tank from the leaves before they fall off.) You can collect and boil these to make tea. Search 'blackwater tanks' or 'blackwater amazon biotope'.
Also, take a look at this site to get an idea of the variety of sources of tannins.
https://tanninaquatics.com
To make soft water from hard water, you need to dilute the hard water with RO or RODI water. If you mix your water with RO or RODI water at a 1 : 1 ratio (half and half) then your GH, KH and TDS will be cut in half. It's that simple. If you mix 75% RO/DI water to 25% tap water then GH, KH and TDS are reduced to ~25%. You must have a GH & KH test kit to do this.
Distilled water is pure water so it would also work to dilute ... But using distilled water just isn't practical ... that's way too many one or five gallon jugs of distilled water.
What size tank are we talking about?