If you go astray, you lose the path. If you went to college intending to become a doctor, but instead became interested in making violent video games, your parents might feel you have gone astray.
A stray animal is one that has gotten loose, and the adverb astray comes from this sense of wandering off from the proper place, even though it's not just for animals. If you go astray, you lose the right way, or are out of place. It can be a moral judgment: juvenile delinquents are kids who have gone astray. But it can also refer to small details. If you dress impeccably, you make sure that no thread is astray.