I’m not really sure exactly how the reputation system works.
My reputation was at the max negative (I believe -50?) in talmberg when i had not stolen or killed anyone.
Even things like lying to npcs, refusing their request, intimidating people and killing surrendered enemies can drop you’re reputation. So if you’re doing any of them that could be part of it.