No, not that I'm aware of. He thought Baker was unprofessional on set (mind you, I've heard stories about Pertwee that would make your eyes melt). I think it's really just a personality clash. In two ordinary people it wouldn't be anything of note. In two of the Luvviest Luvvies of Luvdom, it couldn't help but take on Promethean proportions.
Tom Baker has said he became a bit of a monster towards the end of his tenure, the adulation and stardom will have gone to his head. He's matured and is is now just a bit nutty!