Don't worry about it too bad. It happens to the best of us. I have asthma, it sucks but I live with it. Thats why I never played sports in highschool and played video games instead lol. And inhalers suck to have because there embarassing but they help when their needed.
You won't be allergic to an inhaler sir. Get that out of your head. Your acting like a teenager that just got knocked up and is arresting about what her parents will say lol.