It's possible the cold sores may have been canker sores. Sometimes the two look very similar.
I also have always tested negative for HSV 1 & 2, but did have canker sores when I was younger.
It is my understanding that herpes lesions are a pustule, often like a pus-filled small pimple. Canker sores are like craters. You could have a herpes lesion and just think it's a small zit.
I used to get canker sores when I was younger, and now put it down to a lack of vitamin B, because I was so often dieting and not eating many carbs (wheat products). My sister got canker sores worse than me, and started taking a B supplement and they didn't recur.
So, I went to my health clinic for a STD panel today. I asked for tests for HSV 1 and 2 (besides the other stuff). The doctor tried to talk me out of it, citing that 90% of people worldwide have HSV 1. And she asked me what I would tell new partners if I tested positive. I said, "Never mind all that, I just want to know." Results should be ready in a week.