When you scrimp, you get by on very little money. If you scrimp all year long in order to buy tickets to the Stanley Cup finals, you save every last penny to be able to go to those games.
If you scrimp, yes, you are watching every dime trying to spend as little as you can — for the purpose of saving your money. Maybe you walk everywhere to avoid paying for a bus ride, or you make your friends cut your hair. Scrimp looks like shrink and scrape and that's the goal: shrinking the amount of money you spend to scrape together some savings, for a specific purpose or just the security of having money in the bank.