I would be annoyed at that.
I generally only wear certain colors(black, gray, yellow, blue, green, pink, brown) but I'm trying to expand out more so to red, purple, etc. I've always purposely avoided white for some reason but try to wear more of it now. I think black and yellow look best on me. Yellow is my favorite.