Infinite describes things that are endless, like the universe, or your uncle's corny jokes.
Finite means "something with an end," and when you add the prefix, in- meaning "not," you get infinite: something that never, ever ends. If someone has read every single book about pyramids, you might say he as an infinite knowledge of ancient Egyptian culture, even though that's an exaggeration. He will sure stop talking about them at some point. Right?