No offense to smokers but it does matter. Second-hand smoke can be unhealthy, not to mention the smell of smoke and how it consumes everything it comes in contact with. Relationship wise, smoking is a deal breaker.
Also, why should a non-smoker date a person who has a smoking addiction? Nicotine is a powerful drug. It's not much different from a cocaine or alcohol addiction.
I am a non-smoker and yes I would date a smoker.
My dad has smoked for ever and I have dated smokers. I just make it clear that there is absolutely no smoking inside the house or car.