Jumbo shrimp? Open secret? Use oxymoron to refer to a word or phrase that contradicts itself, usually to create some rhetorical effect.
When Shakespeare's Juliet says, "Parting is such sweet sorrow," she is using an oxymoron; her apparently self-contradictory turn of phrase actually makes a neat kind of sense. Oxymoron is sometimes used to describe a word combination that strikes the listener as humorously contradictory, even if the speaker didn't intend it that way. The word oxymoron is itself an oxymoron; in Greek, oxy- means "sharp" or "wise," while moros means "foolish."