Use the noun polls when you talk about a polling place, or the place voters go on Election Day to cast their ballots.
You can use the term polls for the physical building in which you vote: "I'm off to the polls to vote for dog catcher!" It's also used when people talk about the process of voting itself: "When the polls opened in Mexico this morning, the mood was uncertain." Polls were originally "heads," and later, "people." In the 1620's, the idea of "counting heads" evolved to mean "counting votes."