i have a friend with Red bellies and every time i go over they always hide behind the driftwood and plants,.
this will happen with sunfish too. fish will recognize their owners and normal activities. when there's a new person or unusual occurrences they get nervous.