To do something restlessly is to do it in a uneasy or anxious way. When you're worried about something, you might sleep restlessly, waking several times throughout the night.
Someone who sits restlessly through a boring class might fidget, and a crowd waiting restlessly for a band to come on stage may mill around in an impatient way. When you're restless, you're bored or anxious, and acting restlessly means demonstrating those feelings. The similar Old English word restleas meant "deprived of sleep."