I think this is a fair point to make, but just the same way that people who don't play to win don't enjoy playing to win, it goes the other way too. In general I think there is a belief that people who play to win consider themselves more devoted than those who don't, just due to the time put into the game and learning the trade. However, I don't think this usually becomes an issue until a very pivotal point, that I have never understood.
That is when a casual gamer comes to a tournament, competitive online hub, etc. and is upset at what happens there. It's akin to walking into the lion's den and not expecting to get eaten; when you go to a tournament or something similar, those are the types of people who are going to be there.
JackKeiser mentioned he's seen people get destroyed at tournaments and then get upset. Those people, wether they like it or not, are playing in a competitive arena. If you come to a tournament, people are going to play in a competitive fashion.
Now, one would think that if you don't want to play in a competitive fashion, you just wouldn't come to a tournament right? But somehow a lot of casual or "not playing to win" type people think its a good idea, and then get fed up with people not playing their way.
I feel like if not for those types of incidents, people would be able to co-exist, but so many times I've seen people come to events like this and then be appalled that their opponent used a "cheap move to win." Then they go complain about it, saying how much of a jerk their opponent was.
In my opinion, no player who plays to win goes around looking for casual players and telling them how bad they are. To the contrary, casual players come to tournaments and when they lose, accuse their opponent of being a douche. The agression comes from the casual, not the other way around.