Hello!

I know the official answer will be "You should always use the one that came with your device" - But my new HP Pavilion X2 2 in 1 charges via USB C. It came with a charger that outputs at 5.25V.

My new Nexus 5X also charges via USB C but its' charger outputs at 5v.

I've popped the smaller, 5v charger into the HP and it seems to take the power OK (I'm guessing it might charge a bit slower).

My question is that, if for some reason, I plugged the HPs 5.25v charger into the Nexus = would it 'fry' it?

USB 2.0 spec is 4.75-5.25V
So it really doesn't matter.
You really need to be concerned about current and protocols which have the main bearing on charging speeds, although in some cases the protocols allow for a higher voltage.