In a four person semi final, if the person bottom of the leaderboard is
Bottom of the public vote : they're in the dance off
3rd in the public vote : they're in the dance off
2nd in the public vote : almost certainly in the dance off (but can escape if the public favourite is top of the leaderboard, the public's least favourite is second and the public's third favourite is also 3rd with the judges)
1st in the public vote : safe from the dance-off (unless the public least favourite is third with the judges, second favourite is second with the judges, with the public's third favourite top of the leaderboard)
A little convoluted, but the take-home message is yes, they more or less have to top the public vote unless they get lucky with how everyone else falls. If they top the public vote they're probably safe, but it's not impossible they still fall into the bottom 2.