Sounds like are getting harassed day and night! They may not ever wanna open up. They are gonna stay stressed and keep their doors shut, may eventually starve to death (if they don't get eaten before then).
You are probably right. That was the only time I have seen them picked at during the day though. I was hoping that they wouldn't stay closed up so long and they would just close up if bothered for maybe an hour or two. It seems as if they stay closed for a long time though when they are upset.