After finishing the Robber Baron side quest I returned to the camp after some time and I found out that everyone was alive there. Including the baron.
Same, I just killed him and took his Groschen and shield.
I like his shield however even if you let him live the bandits wont be hostile until you kill one of em than every do many days het respawns allong with the bandits and All charge you do have fun fighting a 15 on 1