Clearly men are allowed to ogle.
Not really. It’s more like complete separation. When I was in Saudi, in one restaurant, I got up to go to the bathroom and started to walk the wrong way – apparently heading towards the women’s section. I was quickly blocked and herded the right way; apparently it’s a pretty serious offense if you’re a single male and go into the women’s section.