Someone who's calculating is scheming to get exactly what they want, no matter the cost to other people. A movie's calculating villain might be determined to steal his innocent victim's money.
The adjective calculating originally meant simply "doing calculations," or in other words, doing math. It quickly took on a figurative meaning that's become its main definition, used for anyone who directs their intelligence and know-how toward ruthlessly getting the best of other people. Your brother's calculating smile is a sure sign he's up to no good.