From the conversation they had during that scene when Daryl's shirt ripped it sounds like Merle left home leaving Daryl there to deal with his father beating him on his own. Like Merle abandoned him, but he said he had to leave or he would have killed him.

He's such a selfish askhole I can't imagine he cared that he left him behind too much; he can claim that whole tight brotherly bond thing, but he's just a big douche bag.

Byron,GA> Charleston, SC> Jacksonville, FL> Guilford, CT> Rohnert Park, CA! A southern drawl in sunny Cali! .
The amount of time from slipping on the peel and landing on the pavement is exactly one bananosecond.
I do have a secret yen for pink in unexpected places. ~ninja dog
I've decided that I'll never get down to my original weight, and I'm OK with that--After all, 8 pounds 2 oz. is just not realistic.