Really? I'm also a white girl who grew up (and lives) in Appalachia and I've always felt that people in my area (southeast KY) are and were more divided by class than race. This is a phrase I've heard before meaning that someone was rich and didn't know what it was like to be a blue-collar worker and so on. I've actually heard a lot of these sort of comments in the McConnell vs. Lunsford race for Senate.
Oh, I've also heard this being used toward people from other areas - of the country, not the world.