"either we all matter or none of us matter."

To whom? I have to ask. The only person I can think of to whom we all would matter or not matter would be, er, 'God'?

We really all do matter to each other, but our senses are dumbed so far down we don't feel it that much.  Just as all the cells in our bodies are part of all the other cells in our bodies, so are all people part of all other people in a greater body called humanity and even bigger called Life. 

Just as a matter of geography, most of us do not matter to most of us. One has to accept that the death of a much loved pet , let alone a much loved family member, would result in more personal grief than the deaths of thousands in another part of the world.

Granted.  But in that big ongoing OM that permeates the universe, there is birth, death, happiness, sorrow, everything.  The closer one is attuned to that, the more one feels of all there is to feel.      

And then of course, not all people are the same. Just type narcissism into google and read the stories of person after person describing the daily cruelties inflicted by these people.

They be sleeping.  When they really truly realize they are doing more harm to themselves than to anyone else, they will evolve.  Sometimes they have to have something happen to shake them out of their sleep, so you want to stand clear.

It seems to me that some people are highly empathic, they develop empathy through the use of their imagination, then there is the normal way, through experience, and then there are those who do not, these are people who derive pleasure from causing pain to another.

That last group is acting out something that they went through in this or another life.  Nobody is born wanting to hurt somebody else, imho.  Compassion and not allowing onesself to become a martyr is the rule with them.     

Personally I came to a point of decision. Does it matter, how everything is, do I matter. The decision I made was yes, that is how it feels for me, it does matter, I matter....to me.

May I suggest for your head to replace the word 'decision' with the word 'choice'.  Words do matter.  And you matter to way more than just yourself. 

And I do matter of course to a small group of people, I have a certain sphere of influence. In fact there are some little tatsoi seedlings right now that owe their life to my care, who are dependent on me to water them or they'll wilt and then die. Whilst I eat their elder brethren...

You are right on track, taking care of plants and the earth and eating the green herbs and fruits bearing seed.  That was our very first job description from the Landlord when we were moved into the Garden of Eden, so to speak.