If someone is penitential, they're remorseful about something they did. Being truly penitential for losing your brother's phone means buying him a new one — not just saying, "Oh, sorry!"
The adjective penitential is less common than its synonym, penitent, but you can use it whenever you're truly contrite. The penitential letter you write your neighbor after your baseball shatters his kitchen window will probably make you both feel better. This word is often used in a religious context, describing people repenting their sins. The Latin root, pænitere, means "cause or feel regret."