I know when I do my bi-annual polish on my car, i use a clay bar to removed all the surface contaminates, so that I am not polishing crap into my paint and making it worse.
A clay bar with some quick detailer, with frequent kneeding of the clay bar inbetween sections would probably be a good way of getting rid of all the dirt stuck to the acrylic.
And if your going to use a buffer, use a D/A. That has a low heat build up compared to rotary, and is also begginer user friendly. Also, you need to look at the cutting of your pads when using a buffer, and make sure you are using the correct one.
When removing a product, use a high qaulity, edgeless microfiber cloth.
Just my two cents. I know acrylic isnt exactly like car paint which is where my experience comes from, but both are easy to mar