The only practical way to remove nitrates is through water changes. Some people find ways to run their water through anaerobic conditions to remove nitrates, but it's fairly complicated, and if done wrong can result in sulfur compounds.
You could also add lots of plants, which will take nitrate if they have to, although they prefer ammonia, so what'll probably happen is that they'll take in all the ammonia and kill off your beneficial bacteria, which won't be a problem if and only if you can keep the plants alive and thriving. Otherwise your tank will recycle.
Don't mess around with the pH too much. Except for the most delicate of fish, pH is fine even if it's not the same as what's in the wild, so long as it's stable. For example, it's better to have a South American fish, which is normally found in acidic waters, in hard, stable water, than in soft water which is tampered with chemically and which can have pH swings at any given moment.