What's the tank size, what kind of fish and how big are they?
Also to be realistic if its for a few hours they should be alright. When I've moved fish I just let them chill in rubbermaids for up to two hours while I was setting things up and aside from the usual moving stress they were fine.
I guess you should be more worried about your beneficial bacteria but I don't have the time to go into that. Hopefully someone else will answer.