Never fear, Connor probably just realized that he needed to go to the bathroom too, so they tried to follow Daddy and ended up turned around.

That or their imaginary friend offered to lead the way :)