A privilege is a special advantage not enjoyed by everyone. If you're very snooty, you probably don't allow just anyone the privilege of being your friend.
Privilege comes from Latin privilegium, meaning a law for just one person, and means a benefit enjoyed by an individual or group beyond what's available to others. Someone wealthy come from privilege. Someone with a library card has borrowing privileges. Privilege can also be used as a verb. If you are on a committee giving away scholarships, you'll have to decide whether to privilege students from poor backgrounds or the students with high test scores.