BB4 was the first season I ever saw, and I think because of that it will always have a special place in my heart. Although it had a lot of boring HM's (most of the ex's), it also had the BRILLIANT Alison! She completely made that season for me. Loved her!
Also, you HAVE to watch BB6. It's probably the most intense season of all. The HM's really started to hate each other by the end, and it's quite exciting to see which -side- will win.
I watched BB2 and BB3 fairly recently, and both of them seem fairly tame by recent standards. I wouldn't really recomend either of them, although I preferred BB3 slightly more.
BB5 is the only season I haven't watched (well, haven't seen the first one either, but that doesn't really count). I'd like to see it though - as I really liked Diane, Jase, and Nakomis in All-stars.