I switched to a closer doctor years ago for this very reason. The pediatrician was so far away, it was awful. It just seemed like the whole afternoon or morning, or both, would be gone.
We actually go to a family practice now and it is so much better! They always work us in. Because it is so close, I sometimes just show up after a rough night and they will see us. I don't dread making appointments anymore. Other things don't have to get canceled because we didn't make it back in time.
The way I figure it, if there is an ongoing medical issue we will probably get referred to a specialist.
I think all you can do is try out a new practice and see if you like it. Good luck.