That's right, if you have marriage contract then there will be no problem. She can still deliver a child in any hospital otherwise, but you won't be able to register the baby for birth certificate and might send you both to the deportation department.