When you infringe on someone's space, time, or rights, you're getting involved in a way that is not cool. That's why, when you violate a copyright, you're said to infringe upon it.
Infringe is almost always used with the prepositions "on" or "upon," as in, "if you infringe on my rights, I'll sue you." Nobody knows why that's the case. Our language is full of rules and traditions that just are the way they are. We call these phrases "idiomatic." To infringe "on" or "upon" something is one of those. You don't infringe "into" someone's conversation. You infringe "upon" it. Case closed.
