Show of Hands

My "friend" is a rude person. He is manipulative and constantly argues with anything I say. Why do I feel bad about cutting ties even though I don't really like him as a person?