Like you, diffusing isn't my favorite method, but the method I use instead is a bit unorthodox and can be unsafe if you don't follow common sense.
I have made a habit of buying tower-shaped space heaters and using that. I make sure to keep it at a SAFE distance (a couple feet) from my hair, shifting my position after each part gets about 75% dry. You can do a reality check every once in a while by touching your hair to make sure it's not overly-hot to the touch. It saves me having to hold something in my hand - frees my hands so I can sit there and read or watch a movie on Netflix
I find the air stream from the space heater is less violent than a diffuser so my hair dries with very limited frizz (if any). Of course my hair type and texture might not match yours, so I'm not sure how much or little frizz it might give you.