I've seen actors get hecklers lol, that's just embarrassing.
I'm not really gonna say much about my reactions on hecklers, it's pretty much been beaten to death; I won't convince you differently, but I wanted to offer some thoughts on your opinion... I guess really the main two issues that people will have with your line is this:
a) What if they can do it
b) It makes the spectator feel bad
a) is admittedly a very small minority. I've never seen it happen to a magician but I have seen it happen to laymen though, and it's not pretty. Just be careful that if you challenge them, they really are heckling you. If I was watching someone perform, no matter how well or badly, I would never heckle a fellow magician. If they were just terrible I'd stick around to talk to them afterwards. Thing is, I could see a situation where a magician was having a bad day or just feeling brash, and if I said something like "nicely done" it could be taken the wrong way. If that magician were then to mistake my reaction as a challenge or a negative response and challenged me, I would most certainly take the deck and perform. I guess basically what I'm saying is just really be careful when you do challenge; I don't think it's wrong to challenge necessarily, but only at the right time. The right time will also depend on the rest of your audience and such factors.
b) I think that however nice you act, however neutral you act, it is inevitable that they will harbour some sort of negative emotion such as embarrassment or anger. As far as the heckler goes, I don't really care if they do feel that way (although, this isn't an issue with me since I don't handle hecklers this way). What's more important here is the attitude of your spectators. It's an option depending on many things, but most of all, your audience. Audience reading is one of the most important skills a magician can have and challenging and embarrassing should only be done in some contexts. If you're only performing for the heckler for example, or if you're performing for a group of blokes who consider each other brothers and would break your arm if you dissed one of them, well, maybe not. If you have a sympathetic audience though that themselves are telling the heckler to stop, a little laugh at their expense probably won't hurt.