if you have high nitrates and 0 ammonia/nitrites, then you need more water changes. i would do 50% every other day until you get the levels low. and yes, i think removing the gravel will help you keep the nitrates in check.
if you do remove, dont do it all at once because you have a lot of bacteria housed in there
if you do remove, dont do it all at once because you have a lot of bacteria housed in there