Hostile Guards

Every now and then I’ll come across a Skirmish with Guards and Bandits on the roads, I watch until I determine who’s who then join in helping the guards, but they turn hostile on me, I don’t hit any of them its always after the battle and I gain reputation even if I kill them all in self defense. Weird.