Friend,
The right time is early morning. The drive to Dubai will take almost 8hrs(including the boarder formalities). Early morning rush at boarder posts will be less... Enjoy your trip....