i dont think you will have problems ... usualy police will not come to ur house if nobody called and said something .Plus if you're the only girl that come to his house neighbours usualy they will not complain ...
AND i have an idea if you are not sure that you want to get married now from him just give the neighbours the impression that you are husband and wife they will not ask you for the marriage papers :P