I'll start this off with a few.
One is that there are complex functions in a given ecosystem that we really are not fully aware of. By releasing a fish into that system it could very well disrupt that.
Some species have the ability to bump out existing species physically or genetically wreaking havoc to the balance of the system. This can have many consequences to both the natural system or in some cases it can have human economic impact.
In some cases differing localities of fish species have very different sets of genes that help them adapt to their particular locality. introducing the genes from one population into another can weaken or destroy the genetic integrity of a specific locality.
One case in study, is the stocking by the Wisconsin DNR of Muskellunge. The stock fish that where being released and stocked were genetically inferior to the "great lakes" strain of fish and set to Grow slow and small rather than grow fast and big. it also turned out that the stocked musky where totally unequipped to compete well when mixed un-naturally with Northern pike as they where adapted to spawning in shallow water rather than deep water. In areas Musky and northern pike naturally co-habitat Musky always spawn in Deeper water. We now know this above information. But 20 years ago we had no clue that this was going on in the genetics or behavior of the fish. In a way a Muskellunge is not just a muskellunge everywhere and they should not be haphazardly thrown about and stocked just anywhere from anywhere just because they are Esox masquinongy .
These days fisheries management officials do think about this and weight it before stocking fish.
In another case, some species of fish have very defined social orders and familial clan ties. When another fish is introduced to this group (even if related and of the same species). It is very likely that the invading fish will be attacked and killed or it will be predated on by being extracted from the protective group. Releasing a fish just because you caught it in the same locality does not mean it will survive or be accepted back by its own kind
Above are just a few of the many things to think about.