You are not alone! Ever since I got rid of my pillow top mattress pad, I've been struggling with this. (I threw it out when I thought I might have bedbugs--fortunately I did not!)
So, in my case, it appears the solution may be to buy another mattress pad! Maybe try that if you haven't already?