Maybe that's what the alleged falling out with the Duke of Edinburgh was about - either talking too much about the Royal Family with potentially incriminating or embarrassing stuff re the above and/or using their name to promote himself/show off/exaggerate his status with them.
Oh? I hadn't heard about a falling out. That would make sense though I suppose. There's also the story about the stolen trout in Glencoe (and the ladies in the french maid outfits) - I imagine Charles wasnt too keen to hear that being told.