So this weird thing happened. I just go visit Inn north of Rattay and when the inkeeper sees me he says something like “who are you?” and then starts to run away! I was chasing him on horse to rattay, he made several turns back and forth and it started to be funny. Then he ran back north down to the woods really far away, so I let him and went back to inn and WTH, he was just standing there like nothing happened and was totally not looking like he just ran 3 kilometers 
When he saw me he got scared again and ran away again! Also some villager in that inn did the same. Others seem to behave normally. I have no idea what happened to those two.
Maybe u have hit him with your horse outside? (accidently)
hmm maybe. But the innkeeper usually stands leaning by the wall. Anyway hitting him or the other guy would make them so mad at me? :-] Does the speed of the horse matter?