I like your idea very much, having abadoned villages and a athmosphere of fear. Everything your said is right, but as a nasty guy I have two little points:
Climate: Around 1400 the weather was comperatively warm, not cold. Not as warm as 100-200 years back, but the so called "little iceage" was only in its very early days.
Please have a look at this: climatic change incl. medieval times
Crop failure: It is one of the very common prejudices against the middle ages, that people were all poor and hungry. I now, that you didn't mean that, but someone could misunderstand that. Surely there were local crop failures and the danger of having not enough to eat in the winter for poor people. But catastrophical famines were rare in that time period. The last big famines in Middle Europe were 1315-17 as far as i know.
There might be local differences, of course.
In the Oposite, food was relatively cheap and the nutrion of the people was comperatively good. For instance: Around 1400 they eat procentual more meat than now and nearly 10 times more than in the 19th cent.! (Hans Jürgen Teuteberg, Günter Wiegelmann, Nahrungsgewohnheiten in der Industrialisierung des 19. Jahrhunderts, LIT Verlag Münster, 1995, ISBN 3825822737)
The abadoned villages were presumerably caused (in the majority) by the "Spätmittelalterliche Agrarkrise" (late medieval agrarian crisis). Short summary: The prices for food were so low and the wages/income so high, that many many farmers were ruined.
The plague: We have to check if the plague ever reached the area where the game is settled. Most people died in the south-west of Europe. But a athmosphere of fear was likely there and a good cause for xenophobia.
Please have a look at this: Spätmittelalterliche Agrarkrise