Depends entirely where you live. It's illegal to co-habitate here, but the enforcement varies enormously. If you are Western, non-Muslim, and you live in a compound, then I cannot fathom you having a problem so long as you let people assume you are married.

The potential problems would be if one of you is already married, or if one of you were to be injured access to one another would be difficult for an unmarried couple.

My advice is if it is short-term (i.e. only for visits and such) then don't worry about it. If you are both planning to be her long-term, then just get married in a quick civil ceremony to have the certificate.