A detractor is someone who puts you down. When you're proposing ideas at work, your detractor is the person who finds fault with everything you say.
Use the noun detractor for someone who is always critical. You might describe your brother as a detractor of the government if he complains incessantly about taxes, voting, the President, and all the members of Congress. If a person takes a dislike to you in particular, he is your own personal detractor. The origin of detractor goes back to the Latin word detrahere, "take down, pull down, or disparage."