Javert is a highly complex character.
He's guided by a sense of morality and certainly with respect to the law sees the world as black and white - you're either a criminal (and thus scum) or not. That's why he's then driven to suicide because he couldn't reconcile Valjean's acts of kindness with his world view.
While he is the primary antagonist, I think he's more a victim of circumstance thanks to his upbringing - "I was born inside a jail". It's almost as if he saw what jail involved and was determined to keep away from it, so went as far the other way as possible.