Hesh, what you need is consistent water conditions. Fiddling with your pH will simply be more damaging in the long run than you anticipate. I would advise against fiddling your pH. If possible, focus on your hardness levels as those are the ones that buffer everything thus stabilizing the water conditions.
What are your pH, KH and GH currently on both tank water and tap water?