Whether you liked that dog or not, he was around for about half of your life. That's quiet a long time. Even if you did not really care about the dog one way or the other, you certainly have gotten used to having him around and now something is missing. Plus I think that his not having been well for some time might have its part in your reaction, too. Maybe a part of your reaction is also relieve. Not about the dog being gone, but about him not having to suffer anymore.
Whatever it is that makes you feel this way, I hope you will finally start to feel better again. If you feel like grieving that dog, do it. I have found that burying our dogs after they died help to handle the grief. If that is not an option in your case, perform a symbolic rite of passing like burning the dogs favourite blanket. That might help you to accept the fact that he's gone.
*sends Lace-Bane her best wishes for dealing with his current feelings*
_________________
Yes, I am serious about that avatar...