A baby is the one wearing diapers and a pacifier in her mouth. That baby is so darn lovable, however, that the word is perhaps overused to describe anyone or anything you feel passionately about.
Everybody loves the baby, so much so that you might call your girlfriend 'baby' even though she's 23. If you work in construction, you might call any woman with nice legs baby as she walks past. You also might snidely call your little brother baby when he cries about getting a ball thrown in his face. When your mother has to take him to the ER for stitches after said wounding, you might defensively say "She's just babying him, he's fine."