I wish someone would explain the Alan Rickman thing. His name gets mentioned alot in these things.
His voice, for one. His manner, for another. 'Sexy' is such a subjective thing. For some people, it's just physical, how they look, and that's enough, but for others (eg. me) it's about far more than that. They have to have something else, a great personality, a talent, an attitude, just... something that elevates them beyond just being physically attractive. Practically all the men I find really sexy are not, on the surface, immediately physically attractive... but they have qualities and/or talents that make them irresistable.
My list includes Javier Bardem, Jeff Bridges, Bill Nighy, Vincent Cassel, Christopher Walken, Bill Murray, to name but a few.