If you have problem with that person..
Why not tell him about it...
Or tell your boss...