to clean or not to clean

That is the question! I was busy doing a water change yesterday when this
question came to mind. I change water every 2 weeks and during the
operation I run my suction hose close to the gravel and suck up loose
leaves, dirt etc that is lying on top. Perhaps I shouldn't be doing this.
What is the word out there? Should one leave the bottom junk for plant

in overcast Vancouver -- but no rain.