I tend to do that too, because I want to cut down on the amount of time my hair takes to diffuse dry. I used to air dry (and still do if I'm not going anywhere, but that takes even longer) but my hair has gotten long enough to where it will take a very long time to dry, and if its air drying it can take all day. So I plop, plop, plop to get the water out, but if you handle your hair too much you can cause alot of frizz, esp. crown frizz. Even if you're using a flour sack towel or a t-shirt, which is what I use.
You could try using paper towels, IMO you do get less frizz and you don't use as much. I don't, I know, because I'm sort of appalled at the waste of paper
It helps me keep my plopping in line!