Whether they keep the passport or not, you still cannot leave the country without an exit permit.. So what does it matter anyway? Let them keep it if they want to because even if it is with you, it is nothing but a useless piece of paper because you definitely cannot use it as a travel document without the permission or exit permit to travel.