If you have sand, the suction on the vacuum will probably be too much and you can't push it down too deep or else you'll be sucking sand.
I have fine gravel in my 120g. I push the gravel vac down into the rocks, the rocks start to move slowly up the tube, usually the top few spin and twirl in the suction. When the rocks reach about half way up the tube, I slowly pull the tube out of the gravel, the rocks in the tube fall back down into the tank. Usually during this whole process, dirt is being sucked out of the rocks. There's a cloud that appears in the gravel vac tube. If your tank is dirty, the darker the cloud.