To destabilize something is to undermine it, or to make it much less stable. Seeming uncertain or confused can destabilize a teacher's ability to control a class.
Political protesters might seek to destabilize a government or other institution, and lying to your best friend can destabilize your relationship. In both cases, the underlying stability of a situation is shaken, or made less sturdy. The earliest use of destabilize was in a physical sense, if something was literally unstable or shaky. By the 1920s, the word came to also have a more political meaning.