A torn bit of old fabric is a rag. You might use a rag to dust the bookshelf or scrub the bathtub, but you wouldn't want to wear one.
If your clothes are torn and dirty, they're also rags, and from the sense of "worthless scrap," trashy or low quality newspapers have also long been called rags. A rag is also a somewhat annoying joke, and to rag someone is to annoy or harass them. The phrase "from rags to riches" describes a classic and deeply satisfying storyline: the tale of a poor person who becomes gloriously rich.