In Series 1, the only reason Anton & Lesley didn't get the worst Latin scores was because Chris Parker was in that series and, bless him, he was horrendously bad at dancing altogether.
In Series 2, Anton & Esther were out in week 2. Again, they didn't receive the worst Latin scores because other celebs - Quentin Wilson and Diarmuid Gavin - were both substantially worse than Esther. Not that she was comfortable with the Latin either, mind you.
I didn't see all of series 3 but Anton & Patsy got quite far. At the beginning of the series, the worst scores went to Fiona Phillips and Dennis Taylor. Once we got to the latter stages of the competition and the poorer contestants had been eliminated, Anton & Patsy had the lowest scores for a few weeks, until they were eliminated themselves. Although that included ballroom as well as Latin.