I use a small round brush when I dry as well. They also make hair dryers with the brush attached, so you only need one hand and they don't require as much coordination.
I use
this one. It is a little loud, but most hair dryers are. And now I don't need to hold the dryer in one hand and the brush in the other!