You're right, Trudi - but that was an early scene in a car and Estes stood him down because they realised they hadn't successfully picked up Abu Nazir so they still needed Brody.
Later - ie, in a later episode - there was a scene in Estes' home where Quinn was sitting quietly waiting for him one night. I can't remember distinctly how he phrased it but he said that if Estes didn't want to find Quinn up close and unpleasantly personal in a dark place one day, he'd lay off Brody. I can't remember if his motivation for that encounter (which I loved!) was explicitly stated.
However, I'm fully prepared for someone to correct me on the precise context as I may be misremembering exact details.
ps: I think all the "spoilers" above are just speculation and don't need to be hidden. Speculation is fun and just part of sharing what we're watching.