Harmful, yes. Unacceptable, no. Making non-smokers breathe in your smoke may be considered unacceptable because you are imposing something on them that is detrimental. As a by-product, you can also argue that making their clothes smell of something they don't like is also unpleasant.

It does differ from racism in so many fundamental aspects, that I'm surprised you can't see them. Secondary smoke in closed areas does cause harm. Therefore it is fair and reasonable to provide a choice for non-smokers to enjoy an evening in smoke-free surroundings. However, I think it is fair to provide facilities in which people can smoke, if they so chose. As I mentioned before, it's not keeping anyone out of either facility - smokers can go and not smoke in non-smoking pubs, and non-smokers can come into smoking pubs. You are arguing that they are *completely* segregated, with no exceptions.
You see, it's a matter of choice whether to smoke, whereas the colour of your skin is not a choice. There are good reasons not to smoke, whereas there are not good reasons for racism.
I think you're getting mixed up in the 'hate' aspect. Smokers and non-smokers don't hate each other as people, and don't look down on each other as human beings. The only reason for a choice of facilities is just that - to provide a *choice*. It doesn't impinge on normal social interaction, and as for the example of violence, that really is spurious.