Use the verb doff to describe removing something. You probably always doff your cap before the playing of "The Star-Spangled Banner."
The word doff and its antonym don date to the mid-14th century. Doff is a contraction of "do (take) off," and don is short for "do (put) on." By 1755, these words were all but obsolete, but they came back into vogue thanks to Sir Walter Scott, author of works like Ivanhoe, Rob Roy, and The Lady of the Lake. The popular Scottish author used them frequently, and he and his readers kept doff and don alive.