Keep in mind... there are millions (or more) teeny tiny particles floating in the water... if you just wait and let them settle they are still in the system. Over time they will swirl back up challenging attempts at "crystal clear" water and ending up in your filter media. Sand, even teeny particles, are rough on media...
While I understand they are so tiny this is all happening on a small scale and isn't that big of a deal. I still feel it is worth the effort to remove them from the system through thoroughly washing the sand first, then water changing as necesarry to remove what is missed.