Can you give a little more info, like has the tank been set up long before adding them or did you add anything to kick start the cycling? Generally you leave your fish in the tank and only do a partial water change, depending on bio load people will do 25-50% on a weekly basis, but that varies greatly. If your tank starts to build up ammonia (as it probably will) youll want to do smaller water changes every couple days or so (and leave the fish in). The goal will be to keep most of the water in the tank to allow the cycle to happen, but remove enough water each time to keep the ammonia at a safe level.
Anyhow besides how long has the tank been set up, what sort of filtration are you running, whats your stock and what are your water parameters (pH, ammonia, nitrites and nitrates). With that info we can probably give more specific help