That was their choice to live in a council house, there's plenty of ways of getting out of that world if you actually try.
He doesn't (didn't) owe them anything.
Has it occurred to you that perhaps they didn't want
to get out of their council house, and that they were perfectly happy there?
And what exactly is 'that world'? All council houses don't have broken fences and windows with a wrecked car in the drive, overgrown gardens and a Rottweiler on a chain you know.