If something's unalarming, it's nothing to get upset about. If your dog's barking late at night doesn't worry you at all, you find it unalarming.
The root of this word is the verb alarm, which means "cause to feel frightened or disturbed." And when something is alarming, it's scary, upsetting, or disturbing. So the adjective unalarming, with that added prefix un-, means the opposite: "Everything's fine! That noise is completely unalarming. It's just my broken smoke detector going off again."