Louis did a lot of things in the short time he managed Girls Aloud that annoyed all of them. Firstly, he never hid the fact that he felt Nadine was the best singer and that she should be given most, if not all of the lead vocals. Plus he was never a particularly hands on manager and was barely involved in the day to day running of the band, so when he eventually did leave them, it didn't make much difference and he wasn't missed.
Also, he said early on in the group's existence that he believed (wrongly) that girl bands have a shorter shelf life than boy bands, and that the band should split after their third album, Chemistry. This annoyed the group no end, and they publicly admitted they didn't like the influence he had as manager. And if there's one thing bitchy old Louis never lets go, it's when someone publicly bad mouths him (see Ronan Keating), and as a result, he'll slate them forever more. Cheryl, as the only member of Girls Aloud who worked on the X Factor and also the most famous member, bears the brunt of this.