Hmm. That is a perspective I never considered. I have many black friends, and have never touched their hair due to the fact that they'd get pretty irritated with me. I think it's also due to social acceptance of what others consider appropriate or not in the sense that I wouldn't want random people coming up and touching me inappropriately, yet I refrain from touching others out of respect.
All in how you look at it I guess. :P