errr. I doubt the styrene killed the fish, unless you didn't mix it properly. Gelcoat is polyester resin with some solid fillers and UV stabilizers.
To answer the question at hand - the gelcoat will be fine for fish, but you are talking about a TON of work to build a mold and then build the tank around that. It sounds relatively easy, but trust me, it will take you a lot of time, and consume a disgusting amount of material. Of course, this depends on the size of tank we're talking about. if it's a 20gal, not so bad... 100+ and you're talkin about a serious undertaking.