i have fine gravel with a fx5 . sucks a fair bit of gravel up but it doesnt seem to cause a problem. ive allways had some gravel/sand in the filters when i change them but it just sits at the bottom of the filter. never been a problem and ive had some other filters for years. . the intakes end about 4 / 5 inches above the gravel / sand but the fish kick it up to the intakes. i think i would be more concerned with blocking the intake / outlet of the filter with the gravel / sand building up in any loops you might have instead of having straight pipe work but as long as the flow is good that shouldnt happen . i have a dip in my pipe work and never had any trouble .
im sure its not ideal to get it in the filters and if you can prevent it then brilliant . but it wont be easy to stop it completely and you could affect the the performace of the filter depending on how you do it .