Glance involves quick contact. When you glance at someone you take a quick peek at them. When a knife blow glances off you, it doesn't penetrate, but hits at an angle.
When you glance at someone, it's often because you don't want to be caught staring. Remember: "Strangers in the night/exchanging glances..." In certain circles, if someone is caught glancing at someone else's girlfriend, he'll find himself in a fist fight, praying only that his beefy opponent will deliver only a glancing blow.