I have the same problem, and I do the same thing (darn ezcema, lol). It never occurred to me that it might be the conditioner. I just bought a bottle of TRESemme NATURALS Silicone-free conditioner with aloe vera avocado. It also says to rinse out but the ingredients are different than the 10 bottles of VO5 Moisture Milks I have stockpiled in my bathroom. I will try it today.
Natural since: 8/2011
Last BC: 1/20/2013
Curl Pattern: 4C
Porosity: High
Density: High
Texture: Fine
Hair Length: <TWA

"...For You formed my inward parts; You covered me in my mother's womb. I will praise You, for I AM FEARFULLY AND WONDERFULLY MADE; Marvelous are Your works, And that my soul knows very well..."