Well, the items you see could be waste or leftover food. They both add to the bio load of the tank and unless you have fish that will consume them, you are leaving them in the tank. Water changes will eventually remove the items after they degrade, but the question is why wait?
Removing them could 1) reduce the overall waste in the tank 2) allow you to wait longer to do a WC or 3) let you change less water when you do a WC. All good things!
And of course it looks lots better and re-affirms the smart choice to get pool filter sand in the first place.