Is it normal that my brother touched in a sexual way when we where kids ?

Well there was almost certainly curiosity involved and in my opinion if an older child initiated it then it was abuse, but probably not at the most serious end of the scale.

I was touched sexually by my older brother when he was about that age and I was a lot younger than you were with your brother, and also only once, but even though I physically enjoyed it I was adversely affected by it well into middle age, at which time I had specific counselling for it. If it's worrying you, I'd suggest you look for some counselling through a Sexual Assault Centre: the fact that you agreed to touch him too doesn't mean you're not adversely affected by it. I find it interesting that you didn't think about it much for many years, because I totally forgot for decades about the incident with my brother and apparently that's not uncommon.

Please don't procrastinate for ever about seeking help: I wish I'd done so much sooner than I did, because I'm sure that if I had done so I wouldn't have stuffed up so many relationships.