If your going to say that your a magician, make sure you have a card trick to show them, or at least a back palming routine, because their going to ask.
I usually reply with a statement suggesting it's something I've developed myself. "I find it's just something I can do with a deck of cards, I just teach myself." It doesn't hurt anyone, and it increases the respect the spectators have for you tenfold. Most of the time, the same goes for magic.
I just usually say practice, and just leave it at that. There's really no need to tell them anymore. We don't need to recruit anymore laymen, and that's all it would do....
But if they really wanted to know what it was......
haha what a diss line....
anyway, personally I would tell them I learned it off a secret magic site. if they press for details, i'd tell them to go here.