I do not gravel vac every change, but... if there's excess buildup, I get rid of it,Are you gravel vacuuming every time you do a water change?
if every other, or every 3rd water change this occurred, it was sucked out.
I pointed influent pipes so detritus would build up in a certain localized areas, so vacuuming it out was a 2 minute job.
In my heavily planted tanks though, I almost never vacuumed.
As far as the nitrate studies claiming nitrate not overtly toxic, most of that research was done by commercial fish grown for the food market, and usually deals with allowing fish to survive in overly crowded conditions short term, until its time to harvest them, and so they look good as filets, or under plastic wrap.
The long term effect of high nitrate in these studies is usually not relevant, to the food trade, but because the aquarist "might" want their fish to look good, and be healthy long term, per needs are different.
How many times do we see oscars and other large older fish turned in to LFSs disfigured by hole in the head?
And its pretty obvious why, to me, because aquarists take less toxic nitrate to mean, harmless.
I believe "if" the long term effects of nitrate were actually studied, the resulting results would be quite different.