I definitely get asked this often, and I am no longer offended. I only get p.o.ed when someone tries to ARGUE with me about it after I tell them I'm not. My dad told me that apparently if you don't have "nappy can'tcha, don'tcha hair" - his words, not mine, meaning you can't do anything and don't want to do anything with it - people will assume you're mixed with something...but I also got this question often when I had a relaxer too.
OP, it's probably a combo of your hair, freckles and red color that makes people ask. When they ask you, just say, "Nope, I'm not!" and keep it moving. No biggie.