Grimy things are so dirty that it takes some scrubbing to clean them. You'll probably feel pretty grimy after a day spent trying to fix the engine of an old car.
When something is grimy, the dirt is ingrained or set in, not just on the surface. Jobs that leave you covered in oil, tar, or dirt are grimy. If you wear gloves to transplant your grandma's flowers, you won't end up with grimy fingernails. The adjective grimy comes from grime, "ingrained dirt," which goes back to a root that means "to smear."