I use natural soap regularly to cleanse hair and scalp. I got a sample of black soap and found it VERY drying. So, that could be part of the issue.
Castor oil and olive oil are both "heavy" oils. If your hair is low porosity, it just may not be able to absorb these heavier oils. My hair doesn't care for straight oil/butter, but likes them in a well formulated product. My hair HATES castor oil, it makes it dry and greasy at the same time. It takes me a few shampoos with a strong surfactant to remove it. You may need to clarify and start with a clean slate.
3a (Corkicelli), highlighted, fine, low porosity
modified CG, since April '07
CG since 3/11/08
HGs: Anything Sevi; Curly Kinks Satin Roots, Curlycue ReNew and Coil Jam; homemade FSG and okra gel; soap bars; UFD Curly Magic; Botanical Spirits Jellies, CJ Repair Me