Ragan has an incredibly condescending attitude, and he talks in therapy speak. He offers instructions to other people about respecting others but he has no respect. He called Rachel a witch a couple of weeks ago, and has repeated that slur, which from a feminist perspective is anti-female hate speech. (This is not to defend Rachel's bringing his sexuality into the fight, but only to say he is no better than her in that respect.) Moreover he denigrated Rachel for being a waitress, and made a disgusting reference to Brendon and Rachel's bed smelling like old cheese. (This in the diary room, weeks ago.)
All in all it's the combination of back-biting and self-righteousness that I find so annoying.