Can't return the bowl from the cave after Saintly Remains

It is impossible to continue playing because the guard in the yard always say that I am a thief, and the following is a never ending loading screen (and I have to use ctrl+alt+supr to close the game process).

Yeah I hit this too, had to go quite far back to undo it.