An infirmary is a hospital, or the area of an institution that's used as a hospital. If you come down with chickenpox at summer camp, you might have to spend a few nights in the camp infirmary.
While you can call a hospital an infirmary, you're most likely to do this if you're British and referring to a specific hospital, like the Glasgow Royal Infirmary. Infirmaries are usually spaces within larger buildings that provide healthcare. Inmates go to the prison infirmary when they're sick, and soldiers get their checkups at the military base infirmary. The earliest infirmaries were in monasteries. The Latin root is infirmus, "weak or frail."