When mechanical media (that main function is "not" really to provide a biofilm substrate for beneficial bacteria) is covered in gunk, that gunk can smother the minor amount of beneficials that it does contain making them weak, and useless.
When the mechanical media is rinsed, it actually allows for the production of more robust colonies to form.
So whether or not a little chlorine washes some of the weak ones away, is of little consequence, because that area is soon repopulated by a more useful population.
And if the tank is fully cycled, there will be beneficial bacteria on all surfaces (walls, sand or gravel substrate, rocks, wood, etc etc) anyway, in proportion to the tanks bio-load.
I must admit I squeeze out my mechanical media (sponges, Poret foam, floss)in buckets of tank water, not because I worry about chlorine, but because I want all that gunk to help fertilize my garden plants.

To help use nutrients like nitrate, I use floating plants, like salvinia in the sumps, encourage algae growth.

and also use terrestrial plants like New Zealand spinach, and basil for nitrate export, along with small daily or every other day water changes. The nitrate level has been undetectable ever since the tank has been set up.
