Things that are posh are swanky, fancy, and high-class. Posh things usually cost a lot of money, too.
If something is dirty, cheap, and shoddy, it will never be accused of being posh. Posh things are elegant and expensive. A posh apartment is gorgeous and classy. A posh restaurant has fancy food and you probably have to dress up to go there. Posh places are luxurious and fancy-schmancy. So it's no surprise that rich people tend to like things and places that are posh.