In Maryland, I've seen folks throw fish they caught in the surf up on the beach, but they all said they planned on eating them. I can't say I see a big problem with that, but growing up being a hunter/fisherman who will catch/kill animals to eat, I have a different view on that.
As far as non-native species go, I have to agree with killing them as you pull them out. In Maryland, three of the most popular non-natives are:
1. The Snakehead
As we all know, this started from someone releasing an aquarium raised fish and so on and so forth. These fish are direct threats to the native animals. IMO, all snakeheads caught wild in the State of MD should be killed to preserve the Chesapeake Bay.
2. Chinese Green Crab
These guys are prevalent in the coastal bays of MD. They were originally sold as a bait for saltwater fishing, and eventually ended up being released. Of course there are no known natural predators for these guys in MD. If you know anything about the Blue Crab, you'll know it's numbers are incredibly low in general, and that they do move to saltwater for overwintering. With these green crabs outcompeting the blue crabs for food, they're damaging an already depleted population of animals. If you've ever eaten blue crab or you're from the state of MD you'll understand my attachment to these animals. Whenever I catch a green crab, I'm sure to kill it and feed it to the seagulls.
3. Eurasian Watermilfoil
This is a species of submerged aquatic vegetation that was introduced to the Bay in the Late 70's to early 80's. This species has suffocated some of the native species of bay grasses, including eel grass (Vallisneria americana) You may recognize this species of live plant, I do believe it makes it's way into the aquarium trade from time to time. The Chesapeake Bay lies along the Atlantic Flyway, which is the migratory route for millions of species of waterfowl every year. Several of these birds, Canvasbacks, Redheads, Bluebills and others, sustained for years on the eel grass. Now that the eel grass is gone, many of these birds have found different routes or have had to switch food sources. The populations have suffered greatly, Canvasbacks especially, which were, at one time, considered a mainstay along the Chesapeake Bay, are rarely seen anymore.
I know it was lengthy, but I definitely cannot agree with releasing any types of non-native species that you may catch.