Yes, absolutely possible. Also equally possible that the child's hair will change to curly at puberty.
My mom (native american straight hair) and curly dad had:
One lifelong wavy (me)
and 3 with stick straight hair as children, who turned curly as teens.

But I think the good analogy is skin tone, like a previous poster noted. Curly + Straight can = Wavy sometimes; and there can be throwbacks, so Wavy + Wavy can = straight and/or curly haired children as well as wavy haired kids.

In complexion this works the same, my man and I are both fair-skinned, but because my heritage is mixed, some of the kids are not. Straight hair can 'hide' like that too

But your kid? I'd put my $ on curly at puberty, personally.
2c / C / iii (FIA)
Med-Coarse, LowPorosity, NormalElast (LCLF)

Shoulder-Length now.

Wash: DevaCare Low Poo
Detangle & Rinseout: Walgreens or Sally's Conditioning Balm (always)
Leave-in: DevaCare OneC (budget choice is GFTN)
Style: ReCoil & DevaCare Arc Angel (budget choices are HESMU or Got2BSpikedUp)