When you do organize something alphabetically, you arrange it in order of the letters of the alphabet. The fiction section of the library sorts books alphabetically by the author's last name.
Dictionaries and encyclopedias are organized alphabetically, so that it's easy to find the word you're looking for. Some home cooks arrange their cookbook collection alphabetically by title, while others might place them on the shelf by color instead. The word alphabetically comes from alphabet, which is rooted in the first two Greek letters — arranged alphabetically, by the way — alpha and beta.