You can call your grandfather a paterfamilias — he is the male head of your family.
The word paterfamilias comes from ancient Roman law. The word itself is Latin, combining pater, "father," and familias, "family." In other words, he was the father of the family — and in Rome, that meant that he had legal rights to everything the family owned and authority over each of its members. These days, the paterfamilias is more likely to simply preside over family reunions and hold grandchildren on his knee.