Filter floss is used by most of us as a "polisher". It is best to use it in conjunction with coarser pads/sponges. This way the coarser media grabs the larger particulate and the filter floss grabs the tiny stuff. Just using filter floss means it is collecting everything on its own, and will be overcome very quickly with crud, resulting in inevitable "bypass" if it is not changed out very very regularly.Maybe someone could suggest a different mech media?.
Because your tray is very shallow, again part of the design flaw imo, you don't really have the luxury to use multi layered and neatly placed varying grades of seperate sponges to give you this layering effect. What a faff trying to get them in a 2" gap, and then closing the tray trying to keep them in place.
Mech filtration is extremely important for two main reasons when you have a sump. Firstly, it's got to be good enough to keep crud from getting through into your bio side (the whole reason of this thread), and secondly, the more obvious one, your mech side is the stage which hopefully makes the water in your display tank look crystal clear, so it's got to be easy to maintain and extremely efficient. Yours is neither, given the design you've got.
You've two options as I see it. Carry on with the way it is, just change out that floss a lot more regular than you do currently, and hope things improve slightly. At the end of the day a little build up of gunk at the bottom of your bio side isn't the end of the world, just unsightly. Or, you can start over with a new sump set up and customise it to your own liking.