Hmm, you should be fine. I have the 8gb gtx 1070 with 8gb of system memory so I guess that could be the difference, but I doubt it. Have you checked your gpu’s resources while playing?
GPU load highest is 99%, GPU memory used max is 2849 MB
What solves the problem is to Use BES freeware as told on steam.
https://www.nexusmods.com/kingdomcomedeliverance/mods/133?tab=description
When KCD’s process is limited by a -2% cpu usage , there are no more freeses.