The nearly impossible puzzle your math teacher gives for extra credit can be called a teaser — a tricky problem to solve.
A mystifying problem is one kind of teaser — another is a person who loves to tease. Your nose-grabbing, practical joking uncle is a teaser, especially if he enjoys gently mocking or making fun of you. Yet another teaser is the kind that opens a TV show — sort of a sneak preview of the story to come. This is closest to the original meaning of teaser, from 1934, "an introductory advertisement or sample."
Definitions of teaser
noun
an advertisement that offers something free in order to arouse customers' interest