Uhh to clarify skald's post, that should only be done IMMEDIATELY if possible. Like, you put the fish in the tank, realize your mistake, and then catch it out within 1 minute or so. Any longer and the fish's body will have already started to adjust to the parameters of the new water. Putting it back in the old water will cause the fish's osmotic processes to reverse, causing additional stress.
NOTE: Fluctuating parameters are MUCH WORSE for a fish than a single jump to new parameters. Most of the time fish are fine without acclimation as long as the parameters of the new water are similar to those of the old water. Professional aquarium maintenance companies rarely acclimate fish for their clients because they are aware of the water quality and there is no time to acclimate during a service visit.
So you're probably better off just leaving it in the tank at this point. If the pH's and temperatures were similar I wouldn't worry too much.
Keep in mind that it's easier for a fish to jump to a higher pH/KH than a lower one, and it's easier to jump to a lower temperature than a higher one.