I really think that dress is meant to worn as a slip under your clothes as the material just looks way too thin to be worn as a dress.
Nude color is hard to wear and you'd have to be a bean-pole to be able to wear it. It emphasises any fat areas you have. The top bit can easily cause muffin overspill and the shortness will magnify any cellulite. If you have the right skin/hair coloring and have no wobbly bits then you could probably make it work.
As for jealousy - I find men can be even more unforgiving when it comes to the female form than women! A group of guys said out loud to a friend of mine that she looked pregnant when she clearly didn't look it. So don't be surprised if you wear that dress and if your body is anything less than perfect, some pig on the street will probably say something derogatory. Heck, I was walking down the street with my sis and her boyfriend and this very good looking women went past, and as soon as she was out of earshot, he complained about why she would wear short shorts when she had cellulite (which wasn't very much)! So if you think only women would get jealous and make nasty remarks...think again!