Of course Matt Smith wasn't the 'worst' Doctor. In my opinion, he was consistently brilliant even though some of the scripts let him down.
If you want to get official about it, there are no 'worst' Doctors anyway. Only ones that aren't well received by the general public.
I don't see Matt Smith falling into that category either, since he was mobbed with knickers thrown at him everywhere he went. Not to mention, loved by the younger viewers.
The only actor that wasn't well received was Colin Baker. That's only because Grade hated the show and tried his best to sabotage it. Having been fortunate enough to meet the man himself, you couldn't ask meet a nicer guy.