When he first entered the show, he was just an evil guy with his own personal agenda which mainly involved killing innocent people. Now, we've gotten to learn so much about him! Evil isn't born, it's made, and we got to see exactly how it played out for him.
He was harassed and possibly abused by his father, and he always felt like an outsider because he had a different father than his siblings. Then he finds out he's a hybrid - the only one in the whole world. This basically brands him as a freak. His own mother went against him, and he would probably have turned out a whole lot different if she'd just trusted him not to abuse his superior hybrid powers.
Loneliness doesn't mean not having friends or family. It means not having anyone that truly loves you and will stand by you no matter what. Klaus has never had that, which is why he creates his hybrids - to feel like he belongs.
He thinks love is a weakness - probably because he's found it to be the downfall of so many people over the years. He doesn't understand the allure of it and chastises Rebekah for chasing after it. To him, love has only ever brought pain and he has grown to resent it, and he wants to protect Rebekah from that - It's his way of being a good brother to her.
He's always felt lonely, and it's this loneliness that he channels into rage that enables him to take people's lives or torture people without feeling remorse. And he's never had any reason to feel otherwise. But with Caroline, other feelings are breaking through to the surface. He's starting to see what it feels like to do something for someone without wanting something in return, or what it's like when that person values you for it.
Caroline brings out the best in him, and he wants to be good for her. But he wants to survive more than anything, which is why he kills his hybrids and kills Tyler's mom. It makes sense, too. It was either kill or be killed, and he was not going to let himself be killed.
Klaus isn't evil, and not just in comparision with the other characters and all the bad things that they've done. He is good and he only does what he has to in order to survive.