A burden or serious concern is an encumbrance. Your being five feet tall could be an encumbrance when it comes to your dream of playing professional basketball.
Something that blocks you from doing what you want to do is an encumbrance, like the rocky soil in your garden that makes it impossible for you to grow tomatoes. A burden or hindrance can also be an encumbrance, the way wearing thick knitted mittens makes it harder for you to dial your cell phone — your mittens are an encumbrance. The word comes from the Latin word incombrare, which means "barricade or obstacle."