It's understood that the removal from the live line up relates to claims about one of the brothers being accused of assault.
"Everyone is in total shock and no one can quite believe it," an X Factor source told the newspaper.
They added: "New information came to light early this afternoon and as a result bosses felt they had no choice but to suspend Brooks Way until further investigation.