1. When they begin producing so much waste that you're doing twice the amount of water changes that you used to do in order to keep nitrates at a safe level.
2. When fish who used to get along perfectly fine begin fighting over territory.
3. When a fish can not easily turn around.
4. When the length of the tank is no longer at least three times the length of the fish's body.
If you want to go ahead and upgrade, there's no reason not to... They won't complain about extra space, that's for sure! But these are pretty good guidelines to go by.