I absolutely love my two kittens, but it took my husband ages to convince me to get a pet, and I do sometimes think I was technically right (though I wouldn't change things for the world).
We live hundreds of miles away from our families and best friends, so used to be away at weekends all the time, and also like to go on holiday a lot (camping etc). I warned him that we wouldn't be able to keep this up if we got cats. He kept saying that they are cats, not dogs (I'd never had cats) and it would be fine....
But I've not visited my mum and dad since we got them 9 months ago, or our friends. Luckily people have came up to visit us, but it doesn't feel fair to expect that all the time. We put them in a cattery once, and I burst into tears in the middle of the cattery office and worried all weekend. I'm going to have to do it again for two days over xmas, and am dreading it, but I really need to visit my family. (They were actually purring when I walked away and left them, and purring when I went to pick them up, so have no reason to think they didn't like it. But still...)
Luckily, we have recently found a good pet sitter that they seem to really like. We went away for a weekend this month, and they didn't seem to notice, which should open things up a bit for us next year.
So I don't regret getting them, but I do worry a lot about the amount of time we spend out of the house (at least a couple of evenings per week, as well as working full time), and the holidays that we are planning. They are happy cats and have each other, so it is irrational, but it still hangs over me a bit. I certainly wouldn't judge someone for finding the responsibility more than they expected.