I would boycott if a celeb committed a crime especially if they get away with it.
How would you know about the crime? If they are found not-guilty, that isn't "getting away with it".
For me it's not the criminal element. It's whether or not they have a personality that spoils the enjoyment of something that is supposed to be entertainment. Obviously in the case of Gary Glitter he has committed criminal offenses, but he is also a horrible person and it's impossible to overlook that and "enjoy" his music on merit alone (assuming you liked it before).
There was a while when the annoying, but not criminal, behaviour of the Gallagher brothers put me right off listening to their music properly. Their singles on the radio were fine, but I couldn't listen to a whole album without thinking about how annoying they were, which also ruled out going to see them live. I don't find them annoying any more - their surliness seems funny to me these days, so it's all fine.
Cheryl Cole's conviction for assault didn't put me off Girls Aloud, because I didn't think it was racially motivated, and I also thought she was young and stupid and it was a one off and the other girls hadn't done anything wrong. In that respect I think it's easier to overlook the criminal or annoying actions of just one member of a group if the remaining ones haven't done anything wrong. With good PR they make sure that people think of other more positive things when they think about their artists.
For someone like Chris Brown, he was only borderline famous in the UK before the assault charge, so it's going to be very hard for him to ever shake it off.