Quick overview...
Driftwood and peat both emit/leach tannic acids. Being an Acid it makes the water more acidic, or having a lower PH reading...
So if your tap water has a PH of 8... and your peat/driftwood leach tannins... the your PH will slowly start to drop from 8... For example, let's say a week later it's at 7.0 and you do a water change (using your 8.0 tap water)... So your water change will quickly bring it to (approx) 7.3... then a week later it's somewhere in the high 6's before the water change and a litle higher after the water change...
Eventually your PH will hit a 'bottom' and will not drop any lower... but when you do a water change it will bump back up a few points and then slowly drop back to the bottom point.
Some sensative fish will not appreciate this slight swing, others will not even notice... But I think it would add some mild stress to most fish, but they'll survive. But, this sort of stuff adds up and will probably shave a year or two off the fish's overall lifespan.
To avoid all of this many people pretreat the 'new water' to lower the PH to the desired level before doing a water change.
Another way to avoid this is simply leave the PH alone, which has been by far the best method I've found. I have very soft water with a lower PH and my CA Cichlids breed in it just fine even though the books say they do best in moderate/hard water with a neutral PH... But I've never had fish thrive/breed in tanks with a swinging PH...
Whatever approach you take... best of luck!