Payback is a way to get even with someone — it's revenge. If your brother shares an unflattering picture of you online, you might get payback with an even worse photo of him.
An act of retaliation, or hurting someone who's hurt you, is one meaning of payback. A small kind of payback might be stealing one of your friend's cookies after she's snagged a few of your French fries; a huge payback is when two warring countries use increasingly deadly weapons against each other. The original meaning of payback is a "financial reward," or "profits from an investment."