If you refer to something, you consult it. To check your spelling, you refer to a dictionary. If you want to know when your spring break starts, you can refer to your class schedule.
Refer is made up of two very useful Latin roots: re- "back" and ferre "carry," and when one thing refers to another, one meaning “carries back” to the other — there’s some kind of connection. "The term “Spaghetti Western” refers to a kind of Western often filmed in Italy by Italian filmmakers, in Italian." If you have a mysterious rash, your doctor might refer you — send you — to an allergist.