^ Well, for me personally, when i have to "clean" my substrate in a planted tank, i get a long air hose and use it like a miniature syphon to slowly vacuum out all the crap that sits on top of the substrate or on top of the plants... seems to the be to most effective method to get rid of small bit of stuff and not suck up the tanks substrate at the same time...
Another method i use is to either repeatedly (but gently) press down on top of the plants or produce a sweeping motion with my hand above the plants to try and get the looser bits of gunk to float up into the water and then syphon it out with a hose...
Both methods are definitely time consuming, but pays off in the end...
-EDIT-
BTW, are you constantly cleaning algae off the glass panels in-between your regular water changes? i.e. you do water changes every Saturday, but Sunday to Friday you keep scrubbing algae?
The reason i ask is because i noticed when i was constantly cleaning algae off my glass without performing a water change, i also started to see a lot more detritus / brown algae build up as well..