With my first I didn't go out much due to the inept first time mom thing and his high needs regarding nursing. With my second I think we went out the third week since my oldest picked up a nasty stomach bug in the hospital. I was tandeming and worried that the kids would pass it between each other but I guess the nursing protected the newborn. Like others mentioned I wore him in the sling a lot but there's not much you can do when the biggest germ magnet is an older sibling that was touching everything and sticking everything in his mouth.
Despite that I don't remember Ryan having much more than a couple of colds (other than Roseola at around 7 months). We also choose not to vax.