I'm guilty of this, there's a couple I know that moved from Brighton to be near their Grandchildren, the husband has a strong accent, I assumed he was a Londoner but he was born in Brighton. I've also met people from Harrow that had no strong accent, I assumed everyone from London sounded more or less the same
Tbh, they probably do sound the same to non Londoners but we can tell the difference in accent between people who live a few miles apart.
I'm from South London and my best mate is from East and, to me, we sound completely different but maybe to others we sound identical.
Mind you, I think all people from Lancashire/Yorkshire sound the same so I'm probably just as guilty.