Should I leave him wen he hurts me even if he is depressed?

My boyfriend is depressed he just left his wife 3 years ago and they has 3 kids. He wont talk about them with me. He also gets mad if i talk to my friends or if i want to hang out with them. But he wont say it. Everytime I want to go do that see them or hang out he will have a melt down threatening to leave give up on himself or something bad like stop talking to me. If I cant talk or call he gets mad. If I don't want to be more physical he gets mad. I don't know if thats wrong but I feel like i don't want that all the time. He says I am perfect yet he will say i have to do more. I don't know if I love him anymore or if i ever did. I know he needs me. But i hate me more and more each day. If i leave him I don't know if Ican forgive myself for hurting him. I don't know what to do.