Right. Repeated ammonia spikes are like kicks in the head. Your fish can only take it for so long before something goes dim. Ammonia scars their gills and they will eventually suffocate, or if their gills dodge the bullet, their liver might shut down. It isn't a question of 'if' they'll die prematurely, but 'how'. A better question to ask yourself is, "What can I do to make sure there is never any ammonia in my tanks?"
And as stated, 'loose' is a very different word than 'lose'.