Yes, absolutely possible. Also equally possible that the child's hair will change to curly at puberty.
My mom (native american straight hair) and curly dad had:
One lifelong wavy (me)
and 3 with stick straight hair as children, who turned curly as teens.
But I think the good analogy is skin tone, like a previous poster noted. Curly + Straight can = Wavy sometimes; and there can be throwbacks, so Wavy + Wavy can = straight and/or curly haired children as well as wavy haired kids.
In complexion this works the same, my man and I are both fair-skinned, but because my heritage is mixed, some of the kids are not. Straight hair can 'hide' like that too
But your kid? I'd put my $ on curly at puberty, personally.