I think very often, we look at others and feel envious about all sorts of things. I think we envy simply because we are not in that same position, or are in a different position (of course save to the extent that the other position we are looking at, does not involve disaster, pain or suffering).
The point is, it is just a matter of perspective. While the one who envies, well, envies; the one who is envied probably views his position a matter-of-factly, and would not think he is being envied.
My grass will always look greener to you; your grass will always look greener to me.
We are all green with envy. But we fail to remember, whether your grass or mine, grass is just, well, green.