If you were just sticking you fingers in the water when doing maintnance that could be a problem, but taking silvers natual agression/boldness level into account, I think the benefits of hand feeding outweigh the dangers. I think, for the fish's well being, I would rather have him be calmer in the presence of my hand (I really dont think he will show much aggression) than have him freak out and dart around the tank... possibly end up jumping out with the lid open, possibly run in to the side of the tank or deco and injure himself. Even if this does make him overly aggressive, I can always wear a glove or somehting to protect my arm. I do feel though, that once the entire hand/arm is in the tank, aro's will no longer associate you with food. Keep in mind I am only talking about aros as I am sure there are MANY fish out there that will attack you arm with fury, especially if they are used to being hand fed. Just my .02
Also, I did not begin trying to hand feed because I thought it was fun. I would rather just toss some food in there and leave him be. Saves me time/effort. I do this because I believe it is in the best interest of my fish. Not saying I'm right, it's just my opinion. If this ever did become a problem, I would not hesitate for a second to stop. Whatever makes my fish happiest is what I want to do.