Regardless to his reaction, its all your fault, you refused him after taking the deposite, more over you informed him on phone which turned harsh, dont mind that your deal is very weak, this is embaressment to him while deposite with you and denied also.None of your steps are in order.Repeat, that never refuse without meeting face to face.Be straight and stop relying on sms or emails in such deals.And if unexpected! now is the time to learn.