River rocks can be just big enough to get stuck in a siphon, meaning every 2 minutes you have to reach in and pull a rock out, or unclog it and start the siphon again. With normal gravel or pea gravel (smaller river rocks) you can just dive right in, its too small to get wedged. With sand, everything sits on the surface.
Comparatively it is not a huge annoyance, but it is one none the less that can be easily avoided.